No announcement yet.

GUIDE: Rendering shadows correctly for non-head-mounted displays

This is a sticky topic.
  • Filter
  • Time
  • Show
Clear All
new posts

  • GUIDE: Rendering shadows correctly for non-head-mounted displays

    RUISCamera always utilizes a custom projection matrix via a C# method, except when it is rendering for head-mounted displays. Because Unity has a bug that breaks shadows for cameras with custom projection matrices, shadows are effectively broken for all non-head-mounted display cameras in RUIS.

    Luckily there is an easy fix:

    1. Download
    2. Import the package into your Unity 5.4 project.
    3. Attach the ReplaceScreenSpaceShadowsShader script next to the Camera components in all the RUISCameras that are rendering for non-head-mounted displays.

    That should be it.

    How to tell if a RUISCamera will be rendering for a non-head-mounted display?
    A. If “Virtual Reality Supported” option is disabled in Unity’s “Player settings”, then all RUISCameras in your scene will render for non-head-mounted displays.
    B. If the Camera component in RUISCamera->CenterCamera has its “Target Eye” field set to “None (Main Display)”.
    C. If a RUISDisplay with "Split Stereo Display" enabled has the field "Attached Camera" link to a RUISCamera.
    D. If a RUISCamera renders broken shadows.

    When you have located all the RUISCameras that are rendering for non-head-mounted displays, place the ReplaceScreenSpaceShadowsShader script on RUISCamera->CenterCamera. For those RUISCameras that are linked with RUISDisplays that have the "Split Stereo Display" enabled, place the script on RUISCamera->LeftCamera/RightCamera gameobjects.

    If you're not sure about any of this, just put the ReplaceScreenSpaceShadowsShader script in all child gameobjects with the name CenterCamera, LeftCamera, or RightCamera :-)


    Thanks to TomTheoriz for tipping me about the shadow patch.
    If the download link expires, check the original thread where it was:


    Fixing shadows in CAVE setups:

    Most RUIS users won't need this, but if you want to render for your own CAVE setup with properly working shadows, then keep on reading.

    In addition to the above process, some extra steps are required for those RUISDisplays that have the "Head Tracked CAVE Display" option enabled and whose "Display Normal Vector" is something else than [0, 0, -1]. RUIS uses the CAVE Display Normal Vector (and the other CAVE Display variables) to calculate an additional modelview transformation that is baked into the projection matrix. Unfortunately Unity's screenspace shadows don't work if the rotation component in that modelview transformation is anything but identity rotation.

    Here is a simple hack to make the shadows work in a traditional CAVE setup where the display walls are arranged in a cube configuration:
    First, open RUISCamera.cs in \RUISunity\Assets\RUIS\Scripts\Display\ and see line 381 (in RUIS 1.10) in method GetProjectionMatricesWithoutKeystoning() where a Vector3[] eyePositions is defined.

    Below that line, add the following:

    for(int i = 0; i < eyePositions.Length; ++i)
    eyePositions[i] = Quaternion.Inverse(transform.localRotation) * (eyePositions[i]); // HACK
    Now open the CaveExample scene in \RUISunity\Assets\RUIS\Examples\CaveExample\ where you should add the ReplaceScreenSpaceShadowsShader scripts as described above.

    Modify all the RUISDisplay components in RUIS->DisplayManager's child gameobjects, so that they have the following settings:
    Resolution X/Y: This is up to you
    Display Width/Height: 3 (or whatever the side length of your CAVE setup is)
    Display Center Position: [0, 0, 1.5] (half of the side length)
    Display Normal Vector: [0, 0, -1]
    Display Up Vector: [0, 0, -1]

    By default the CaveExample scene has only three RUISDisplays (Front/Left/Right Display). You can keep adding new displays using RUISDisplayManager's "Add Display" button, until you have as many RUISDisplays as your physical CAVE setup has display walls. Make sure that each added RUISDisplay has the same settings as the existing RUISDisplays in the CaveExample scene: each should have the "Head Tracked CAVE Display" option enabled, and have the same Resolution and Display settings (as discussed above), and have the "CAVE Head Tracker" field set to the "Head Tracker" gameobject

    Each new RUISDisplay also needs a corresponding RUISCamera, which you can add by duplicating the existing ones that are parented under the "Cameras" gameobject. Remember to link the new RUISCameras to each added RUISDisplay, using the latter's "Attached Camera" field.

    Here comes the important step: change the rotation of each gameobject (parented under "Cameras") that has a RUISCamera so that they face their corresponding wall:
    Front: [0, 0, 0]
    Left: [0, 270, 0]
    Right: [0, 90, 0]
    Rear: [0, 180, 0]
    Floor: [90, 0, 0]
    Ceiling: [270, 0, 0]

    Each RUISCamera's Transform position should be zero and scale one.

    To wrap things up:
    In "Fake Head Tracking Source" gameObject's Transform component, set position to [0, 0, 0]. Alternatively, you can disable the "Shift Viewpoint With Keyboard" script, and make that gameobject obtain its localPosition from some other source, like a Vive controller (or whatever you preferred head tracker is).

    In "Cameras" gameObject's Transform component, set position to [0, 1.5, 0] (half of the CAVE wall display's heigh). This is only so that the virtual floor of the scene matches your physical setup's floor, and otherwise it doesn't matter what the actual value is.

    If you run the scene you can press the IJKLUO keys to affect the "Fake Head Tracking Source" gameobject's position, and see how the asymmetric projection matrices do their magic, while the shadows will be rendered properly for each display wall.

    Couple of things to note regarding this quick-fix:
    1. The tracking system origin has to be in the center of your CAVE setup "cube". You need to modify your head tracker's input to match that assertion.
    2. The whole thing will break if you apply any rotation to the "Cameras" gameObject, but translation is fine. If you don't have Floor and Ceiling displays, then rotation around Y-axis is ok. As an alternative, you could rotate the whole scene around the cameras, or find a better 'hack' than the above addition of two lines of code.
    3. Your CAVE setup doesn't necessarily have to be a cube, it can also be a rectangular cuboid. Just modify the Display Width/Height and Display Center Position for each RUISDisplay with care, to create a rectangular cuboid of your own liking.


  • #2
    Here is a quick 4-wall "panorama" CAVE test with working shadows that I did by using the method described above. I just added a "Rear Display" RUISDisplay (and a RUISCamera) in the scene and modified the parameters.