Torque 3D Now Supports the Oculus Rift

Torque 3D Now Supports the Oculus Rift

Back on September 20, 2012, we launched the MIT licensed version of Torque 3D on GitHub. Next on December 19, 2012, we launched the first new release under open source, version 2.0. Then on January 10, 2013, the T3D Steering Committee announced the roadmap for version 3.0. Within this roadmap was improved support for various input devices to take advantage of current and future controllers that are entering the PC market.

The first new input device was the Leap Motion with support added to the development branch as announced in my blog on January 24, 2013. The second new input device was the Razer Hydra, whose support was also added to the development branch as announced in my blog on February 22, 2013.

Today we announce support for another device, the Oculus Rift Development Kit in Torque 3D’s development branch on GitHub.

The Oculus Rift and Torque 3D

The Oculus Rift Development Kit (I’ll just call it the Rift from now on) is both a head mounted display and a head tracking system. If you’re not familiar with the Rift you should head over to their site where you may read more about it and watch some videos of the Rift in action.


The Oculus Rift Development Kit

I recently wrote a blog talking about my first steps with integrating the Rift and Torque 3D that you may wish to read: Torque 3D and Oculus Rift (A Tale in Pictures). The two components of the Rift -- the display and head tracking -- each required changes to how Torque 3D operates. Let’s step through this work.


Head Mounted Display
The Rift fits over your head and provides a separate image for each eye. This allows you to feel like you are in the game as the character and provides true stereo vision. Physically the Rift has a single LCD panel that is virtually split in two so that each eye receives half of the display. For the currently available Development Kit (intended for developers to start building VR applications) this works out to a resolution of 640x800 per eye. This is known as side-by-side stereo.

Torque 3D’s graphics pipeline was not originally created with stereo rendering in mind. This meant that hooks had to be added into the pipeline to support side-by-side rendering, which effectively means that you end up rendering the world twice for each frame. In order to achieve true stereo rendering this also means shifting the camera’s world position twice per frame to account for the user’s eye position.


Side-by-side stereo rendering

Something else that needed to be taken into account is that the user’s eye is not centered on each half of the Rift’s display. While wearing the Rift the user’s eyes are shifted towards the center of the whole unit. This means that the left eye is slightly right of the display half’s center, and the right eye is slightly left of the display half’s center. In order for the world’s perspective to appear to converge at the user’s eyes a projection offset needed to be added to each render (offset twice per frame for stereo rendering). This isn’t something Torque 3D was originally setup to handle.

Most people use Torque 3D’s Advanced Lighting mode, which is known as deferred rendering by many engines (although in T3D’s case it is actually light pre-pass rendering). With this type of lighting there are special cases that don’t go through the usual world-view-projection matrix setup, and therefore don’t take a projection offset into account. An example are sun shadows.

And beyond Advanced Lighting there are other special cases that didn’t take multiple viewport (stereo) rendering into account, didn’t work with a projection offset, or both. Examples include spot and point lights, decals, lens flares, light rays, and post effects in general. You can see some examples of working through these issues in my previous blog.

With stereo rendering and projection offset in place, the Rift requires one last step. To provide a large field of view the Rift uses optics to make it look like the display is wrapping around you. To ensure that the rendering out of Torque 3D looks correct through the Rift’s lenses we need to perform the opposite distortion as the last step in creating a single frame. A barrel distortion shader post effect is applied over the entire screen just before the frame is presented to the Rift.


Final image as sent to the Rift


Head Tracking
Having an intimate view of the world makes you feel like you are personally there. But to really sell the simulation you need to be able to freely look around. The Rift provides real-time rotational tracking of your head that we may use in Torque 3D to modify where the camera is looking.

To receive the Rift’s head tracking data we need to include the OculusVR SDK in Torque 3D. This is done by including a new oculusVR module and re-running the T3D project generator. This also includes a number of new support files under the platform/input/oculusVR directory with your project.

With this release Torque 3D supports two different types of input events that may be used with action maps to perform an action. The first is a standard angled axis event that represents the head’s absolute rotation as a vector and an angle. This is the same type of event as is generated by the Leap Motion controller and Razer Hydra controller implementations. The advantage of this input event is it is very easy to work with using T3D’s standard functions, and you don’t need to be concerned with gimbal lock.

The second type of input event uses Euler angles to represent the head’s absolute rotation. These heading, pitch and bank angles are generated directly by the OculusVR SDK and passed along to your code. The primary reason to use this event type is if you want to mix in some rotation values from another device, such as a gamepad. Very often we want to add in a relative rotation from a gamepad’s thumb stick to the Rift’s absolute rotation. This allows the user to spin all the way around within the virtual world without having to actually rotate their head and body all of the way. Using Euler angles makes this very easy to do.


Game Object Support for Head Tracking
Torque 3D already supports rotating a camera or player based on the relative values from a mouse or gamepad. However, there was no concept of an absolute rotation, especially in combination with a relative input.

By expanding upon the new ExtendedMove class that was first introduced with the Leap Motion controller integration, both the Camera and Player classes now support an absolute rotation. By feeding both the Rift’s head tracking and a mouse or gamepad’s relative rotational input into the ExtendedMove input pipeline we now have complete control over the Rift’s view into a Torque 3D world.

With the Camera class this means that you may easily fly around the world in Torque 3D. Ever wanted to feel like Superman?


Flying around the world with the Camera class

With the Player class its 3rd person camera is also supported. This is a really interesting mode as your head rotation has the camera rotate about the player rather than in place. This actually is quite cool and not as disorienting as you would first expect.


Looking to the side of the player from 3rd person

Another aspect of camera rotation that Torque 3D didn’t originally deal with was a bank or roll. Normally the Camera or Player classes ignore all roll input and the view is always on the same plane. It is amazing how quick you become motion/VR sick when you tilt your head towards your shoulder and the horizon comes with you.

To allow for camera banking a new cameraCanRoll property has been added to the ShapeBaseData datablock. When set to true the Camera and Player classes now take banking into account.

When using the 1st person mounted shapes with the Player class (such as the arms and weapons included with the soldier actor), these mounted shapes are attached to the player’s eye and will roll with the view. This has the feeling that the player is bending at the waist rather than the neck. To keep the mounted shapes level with the ground while still allowing the view to roll a new mountedImagesBank property has been added to the ShapeBaseData datablock. When set to false (the default) then the mounted shapes remain fixed in the world while the camera banks.


Tilting your head while the player’s arms remain where they should

Setting Up Your Project for the Oculus Rift

While a lot of changes and new code have been required to make Torque 3D and the Rift play nice together, it is actually quite simple to get started with using the Rift in your own project. It requires that you call some helper TorqueScript functions, and setup the appropriate action map bindings.

All of the details on working with the Rift may be found on our GitHub Wiki Page. I’m sure these pages will continue to grow as people start to use their Rift’s with Torque 3D.

Oculus Rift and Torque 3D Demo Game: Rift Valley

Following our tradition of creating a game demo to show off the latest supported device, Ron Kapaun and I have created a new example game called Rift Valley. In this game you wander around a secluded valley exploring its forgotten structures.



Ron did an amazing job with this demo. All of the art you see in-game he created specifically for the demo (built over the past four weeks) and anyone may use the included art as they like. It is safe to say that Ron is now our most experienced virtual reality artist and I’m sure he would be happy to answer your art related questions. You may check out his other work on his Three Tier Digital Studios site.

When you first start the demo you are presented with some options for how it will run on the Rift. By default, when you press the Play button the game will switch to the Rift’s display at the correct resolution. There are also some experimental resolution modes you may try out. More details may be found by clicking on the About button. Also, be sure to wear headphones while playing the demo. We have 3D sound!



Once you are in the game you may look around with your head. You may also use either the WASD keys and a mouse, or a standard Xbox360 gamepad to move and look around. Using the gamepad has some additional features that you may read about on the About window.


Follow the path and see what awaits


Be sure to check out both the outdoors and indoors

This demo has been optimized to try and maintain less than 16mspf (milliseconds per frame). The majority of this optimization has been with the art that Ron produced to get the most out of the Rift’s resolution, and the need to render twice for a stereo view. This will, of course, depend on your own machine. But we have tested with computers and video cards that are from the 2008/2009 era and appear to have hit our target.

The Rift Valley demo is available for anyone to try out, although you really need a Rift to take advantage of it. Download the demo here:


Rift Valley Demo 1.0


Just unzip the package and launch the executable. Be sure to have the Rift already plugged into your machine before starting the demo. And for those that have come to this blog that are new to Torque 3D, you will also need to make sure the DirectX 9 runtime is installed.

Known Issues

A couple of issues have come up while working on the Rift Valley demo. The first is that the soft edges of the sun’s shadows appear to swim as you rotate your view. This has actually always been the case with T3D’s shadows but is much less noticeable when you don’t have your own head bobbing around the scene. In addition, the sun’s soft shadow edges may also appear to float slightly above the surface they are on, but that seems to be view dependant.

The second issue is with the Rift Valley demo itself. It appears that decals are not working on the demo’s level. When firing the weapon you should be able to see the bullet hole. As decals in general have been tested as working with the Rift, this is specific to Rift Valley.

These issues will be investigated for the next release.

The Future

To paraphrase the Oculus folks, this is Day 0 of Torque 3D’s Rift support. There are a number of improvements that could be made, and I’m sure as people start using Torque 3D and the Rift more things will come up. Here are some future improvements that are on my radar:

Side-By-Side Rendering
The location where the side-by-side rendering occurs could be moved even further down into Torque 3D’s graphics pipeline. Right now all light shadows are calculated once, but general scene culling is done twice according each eye’s view frustum. The ideal situation is to cull once with a slightly larger frustum and then render.

Latency Reduction
There are a couple of strategies that could be implemented to help reduce latency. John Carmack put together a blog outlining the issues along with some possible solutions here. Two strategies that I think could be put into Torque 3D include preventing GPU buffering, and view bypass.

GUI Support
None of Torque 3D’s GUI controls are support by the Rift. If you take a look at the script of the Rift Valley demo you’ll see that I faked things with the Loading screen. Some sort of GUI viewing while wearing the Rift is required by most games.


These are just the first steps at supporting the Oculus Rift Development Kit. Any community input would be much appreciated.

- Dave

UPDATE: This announcement has made front page news on Meant to be Seen. Read all about it!.

UPDATE: Cymatic Bruce reviews our Rift Valley and Aurora Borealis demos:
www.youtube.com/watch?v=yw85AIG1Y4k