No announcement yet.

How do I scan motion of multi user clearly with kinect v2?

  • Filter
  • Time
  • Show
Clear All
new posts

  • How do I scan motion of multi user clearly with kinect v2?

    I have some problem...
    I made some kinect-interactive scene with RUIS 1.082 & Unity3D 5.1.4f1 & Window 8.1
    It used to kinect v2 device and reflected only one user's motion on my character.

    But, some times... Motion tracking is confused over
    while a few people try this scene play all together at the same time.

    character's body is moved follow index 0 player and arms is moved follow index 1 player...or some time, character is not moved.
    It's very weird and ashamed..

    How can i solve this problem?
    Last edited by arms00; 01-28-2016, 10:07 AM.

  • #2

    That sounds strange.

    How many characters do you have in the scene?

    How many people are in front of the Kinect?

    Kinect can get confused if there are many people. But in our own tests with 2 characters and 3 people (one person is "extra") everything has worked fine.

    Kinect 2 works like this: The first person (A) it sees gets index 0. If a second person (B) comes into Kinect's view, he gets index 1. If A leaves and only B is visible to Kinect, person B still has index 1. If your character is animated by index 0, then the character will not move until person A comes back, or until Kinect sees no one, and somebody comes back.

    If there are more than 2 people visible to Kinect, its behaviour could be more complicated.


    • #3

      There is 1 single character in the scene(player index 0 defined).
      And freezing or confusing come about mainly 4~6 people in front of the Kinect.

      Can I take enforced kinect(v2) focus on only 1 closest player
      or setting scanable camera area ?
      Last edited by arms00; 01-28-2016, 11:40 PM.


      • #4
        Yeah 4-6 people is quite many people and you can expect suboptimal performance from Kinect. We always setup our demos so that the background is clear from people, and spectators are on the side. If Kinect starts tracking the "wrong" person, you can reset the tracking by blocking the Kinect sensor with your hand for a few seconds.

        You could create a script that constantly changes the playerId to the closest person. I haven't tried how well that would work.

        Your script should observe the RUISSkeletonManager script attached to RUIS->InputManager->SkeletonManager gameObject. Specifically the Skeleton instances in RUISSkeletonManager.skeletons [bodyTrackingDeviceID, playerId]
        To get the closest tracked person, compare the values at skeletons[bodyTrackingDeviceID, playerId].root.position.z

        You should only compare those instances where bodyTrackingDeviceID == 1 && skeletons[bodyTrackingDeviceID, playerId].isTracking == true

        When you have the closest playerId, change the RUISSkeletonController.playerId (the RUISSkeletonController component that is attached to your character).

        If you are using MecanimBlendedCharacter prefab then you have to take into account the following things:
        * You need to also change the MecanimBlendedCharacter RUISCharacterController.kinectPlayerId
        * You need to also change the MecanimBlendedCharacter->StabilizingCollider RUISCharacterStabilizingCollider.playerId (playerID is private member, change it to public).
        * After starting your game, MecanimBlendedCharacter->Constructor gameObject will be split into three gameObjects: Constructor, ConstructorKinect, ConstructorMecanim.
        * The RUISSkeletonController component is in ConstructorKinect gameObject. Change its playerID.


        • #5
          thank you to reply.

          I'll try that


          • #6
            Can you show more detail on the method to find the player id and current player position of certain skeleton with closest range?

            I tried


            public class ShowRootPosition : MonoBehaviour {

            public RUISSkeletonManager.Skeleton skeleton = new RUISSkeletonManager.Skeleton();
            public RUISSkeletonManager skeletonManager;
            public int playerId = 0;
            private int _bodyTrackingDeviceID = RUISSkeletonManager.kinect2SensorID;
            public float dis = 0.0f;

            // Use this for initialization
            void Start () {


            // Update is called once per frame
            void Update () {
            dis = skeletonManager.skeletons[this._bodyTrackingDeviceID, playerId].root.position.z;



            for getting the value of the player position ONLY but with only 0 in return, not to even mention how to get the specified player id or any extra restriction like distance...


            • #7
              Hi JackPan,

              It looks like you forgot to check if the skeleton is tracked:

              if(skeletons[this._bodyTrackingDeviceID, playerId].isTracking)
                  dis = skeletonManager.skeletons[this._bodyTrackingDeviceID, playerId].root.position.z;
              By default all the position variable Vectors are initiated with 0 values.

              On every frame you need to loop through all the skeletons, and compare their root.position.z values. Something like this:

              float minZ = float.MaxValue;
              int minPlayerID = -1;
              for(int i=0; i<6; ++i)
                  if(skeletons[this._bodyTrackingDeviceID, i].isTracking) // just to be sure, you could add: && skeletonManager.skeletons[this._bodyTrackingDeviceID, i].root.position.z != 0
                      dis = skeletonManager.skeletons[this._bodyTrackingDeviceID, i].root.position.z;
                      if(dis < minZ)
                          minZ = dis;
                          minPlayerID = i;
              if(minPlayerID >= 0)
                  Debug.Log("At the moment the closest tracked person has skeletonID " + minPlayerID);
                  // Change skeletonController.playerId to minPlayerID
              I haven't tried the above code but you should get the idea from it.

              Furthermore, there should be no need for the public RUISSkeletonManager.Skeleton skeleton = new RUISSkeletonManager.Skeleton();
              Instead, you should have public RUISSkeletonController skeletonController; that links to the RUISSkeletonController component in your scene.

              When you're doing this hack of modifying the RUISSkeletonController.playerID in run-time, there are couple of other things to consider:
              1. If you use MecanimBlendedCharacter prefab (which I don't recommend in this case), you need to check out all the other scripts where skeletonController.playerId is used: RUISCharacterController, RUISCharacterStabilizingCollider, and RUISKinectAndMecanimCombiner. The first two scripts require modifying because they copy the skeletonController.playerId value at their respective Start()-methods.
              2. You should only modify the RUISSkeletonController.playerId variable, and not touch any of the code with the local variable playerID in RUISSkeletonManager Update()-method.
              3. Finally, it might be necessary to disable the "Filter Rotations" option in the RUISSkeletonController component. This is because in the current implementation the filtered values overwrite the raw values in skeletonManager.skeletons[bodyTrackingDeviceID, playerId] (which is not ideal, I know).


              • #8
                Ok I'll try and report back!


                • #9
                  I tried your code, it worked. However I still not quite understand how to successfully change skeletonController.playerId in runtime. Should I change every playerId variables that was in Start function into Update or LateUpdate function in the two scripts you mentioned that I should modify? Or should I just add playerId = showrootposition.minPlayerID in every Update function???


                  • #10
                    Normally, the purpose of RUISSkeletonController's playerId is to designate a specific user, so that for example playerId==0 refers always to the same person (the first one seen by Kinect). This normal operation depends on Kinect tracking software's ability to "lock on" to individual users, and persistently keep tracking them if they step out of Kinect's view. If there are more than two people this doesn't work very persistently.

                    In your case, you want the avatar to be always animated by the person who is closest to Kinect. Therefore you need to update the RUISSkeletonController's playerId variable in realtime in the Update() method of RUISSkeletonController, in case that the user (playerId) closest to Kinect has changed.

                    Are you using the MecanimBlendedCharacter prefab? If you aren't using it, then you DON'T need to modify the other scripts (RUISCharacterController, RUISCharacterStabilizingCollider, and RUISKinectAndMecanimCombiner), which are attached to the MecanimBlendedCharacter prefab.