Announcement

Collapse
No announcement yet.

Kinect 2 + Oculus + Unity 5

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Kinect 2 + Oculus + Unity 5

    Hi,

    First of all I would like to thank you for this great tool. It is very helpful. I am using Unity 5 with Kinect 2 and Oculus DK2. Is the Oculus example supposed to work with Kinect 2? When I run the example, the Kinect 2 tracking does not work. Neither within Unity, nor when the application is built.

    Additionally, Kinect 2 tracking works fine in its own example, where I can make the avatar walk etc. I also have the similar problem as the other forum member that Kinect 2 tracking only works within Unity but not when the project is built. I will try the suggested solutions to see if it works.

    Thanks.

  • #2
    I'm having the same issue.
    I can't seem to get the kinect to work with the dk2. Both are recognized and I can calibrate them when pressing esc, but whenever I finish calibration the software returns to the game without any kinect response. Very weird bc calibration mode does showcase the kinect and dk2 working and I am able to calibrate. After calibrating successfully the calibration mode doesn't seem to save my config so whenever I go back to the game and then back to the config the kinect 2 never is saved as a checkbox, The dk2 is detected though and working. Any thoughts?

    I've dug thru the user guide and thin I'm doing things properly. I also was wondering if anyone might know a way to display the kinect point cloud back onto the user instead of an avatar, and if anyone has had any luck including the leap into this. Thanks for your help and a great unity package! Cheers

    Comment


    • #3
      sakahn,
      In the Oculus example you need to turn on the "Kinect 2 Enabled" toggle from the RUISInputManager component that is attached to RUIS->InputManager gameObject.
      Regarding Kinect not working in the build; did you use the following guide to update RUIS 1.07 to Unity 5?
      http://blog.ruisystem.net/ruis/using-ruis-in-unity-5/

      R2d2,
      Are you accessing the RUIS menu in Unity Editor or in your build? The menu saves settings into inputConfig.xml but they are ignored by default when you run your app in Unity Editor. Instead RUIS uses the settings of RUISInputManager component (located at RUIS->InputManager gameObject). So when working in Unity Editor you should enable your devices from there.

      Comment


      • #4
        R2d2,
        About the Kinect point cloud: If you are using Kinect 2, then the default Unity example by Microsoft does this already. See the MainScene.unity in Assets/Kinect/ folder. I'm not 100% sure that it works though, because the scene might have been modified by my assistant when he was testing Kinect 2. If it doesn't work, you could try downloading a fresh Kinect 2 Unity package from Kinect SDK 2.0 website.
        I haven't tried combining Leap Motion Unity package with RUIS yet, but in principle this shouldn't be too hard, unless Leap does something funny with the Oculus scripts. Currently the Oculus Rift C# scripts that come with RUIS are from Oculus 0.4.4-beta Unity integration package, so that _might_ cause conflicts if the latest Leap Motion Unity integration package depends on newer Oculus Rift C# scripts.

        Comment


        • #5
          Originally posted by ttakala View Post
          sakahn,
          In the Oculus example you need to turn on the "Kinect 2 Enabled" toggle from the RUISInputManager component that is attached to RUIS->InputManager gameObject.
          Regarding Kinect not working in the build; did you use the following guide to update RUIS 1.07 to Unity 5?
          http://blog.ruisystem.net/ruis/using-ruis-in-unity-5/
          Thanks for the suggestions. I didn't toggle the RUISInputManager and was also building the app in the root folder. Once both issues are taken care of, I can see both Kinect and Oculus working together.

          If I understand correctly, and based on my experience after using the app, the main actor that is being viewed by Oculus is not connected to Kinect. The actor controlled by Kinect is standing on the side, and one can view it by turning the head right. Is this the expected behavior? May be the solution would be obvious, but do you think it is possible to combine the Oculus actor with Kinect in such a way that they are the single actors, and as you move your hands or the legs, it feels like that you are actually walking in the game? Thanks a lot.

          Comment


          • #6
            Originally posted by sakahn View Post
            do you think it is possible to combine the Oculus actor with Kinect in such a way that they are the single actors, and as you move your hands or the legs, it feels like that you are actually walking in the game?
            Yes, that's one of the cool things that you can do with RUIS :-)

            There are two ways of achieving this.

            1) Use Kinect's jittery head-tracking that can track the whole tracking range of Kinect.
            I assume that you are using the MecanimBlendedCharacter prefab (because it's in the OculusRiftExample scene). Disable the 'OculusDK2HeadTracker' gameobject that you can find parented under MecanimBlendedCharacter->HeadTrackers. After that RUIS will use one of the Kinect head trackers.

            Please note that there is a following issue with this scheme; If the Oculus Rift head-mounted display comes in to the view of the DK2 camera or leaves it, a noticeable "jump" can be seen in the virtual camera position for the duration of one frame. You can circumvent this by pointing the DK2 Camera downwards about 50 degrees, so that it can't see your head when you are playing.

            If the Oculus Rift rotation seems misaligned with Kinect, then read the following: Oculus Rift rotation drift correction and setting the rotation forward direction are done using the DK2 Camera. This means that the DK2 Camera should see the Rift HMD at least when you first start your application. It also means that Kinect and DK2 Camera should be facing the same direction, with the latter pointing downwards about 50 degrees or so as mentioned above. If you are using DK1 then Return key can be used to reset the Oculus forward direction.

            2) Use DK2 Camera's accurate head-tracking with the caveat that it tracks a smaller area than Kinect.
            Place the DK2 Camera above Kinect, so that it is looking in the same direction as Kinect. Both sensors should see your head when you are using your app. You still need to calibrate the coordinate system between the two sensors. Calibration needs to be performed only once, but you have to do it again if you move either one of the calibrated devices

            You can do this calibration within Unity Editor or your build, by pressing ESC in run-time, choosing "Kinect 2 - Oculus DK2" from the Device Calibration dropdown menu, and clicking "Calibrate Device(s)". During the calibration procedure you are asked to hold the Rift HMD in your right hand. The calibration is a pretty straightforward process, but for best results it is important that you move your right hand so that the "calibration samples" are collected from a full 3D volume (sphere, cloud etc). In other words, don't just make circles or other hand motions that result in a planar collection of samples. If you need further info, there are some other calibration details in the RUIS readme's section "Calibrating two different motion trackers to use the same coordinate system".

            Comment

            Working...
            X