Deferred Shading
by Andrew Mac · in Torque 3D Professional · 06/22/2014 (9:38 am) · 116 replies
About 2 weeks ago Azaezel, Timmy and myself set out to bring Physical Based Shading to Torque. One of the biggest hurdles we ran into is the lighting information we need is not available at the stages we need it. This is due to Torque's prepass lighting (aka deferred lighting).
How does deferred lighting work?
Everything is rendered twice. Once to calculate lighting, and then everything is rendered again + the previously obtained lighting information for your final lit scene. This is why a 30k poly model loaded into Torque will show 60k.
This may sound very expensive but it's made in a way that the light prepass is as cheap as possible. It's by no means a bad method of doing things, nor was it a poor choice at the time. In fact, most engines around that time frame used deferred lighting. It also has the advantage of running better on older hardware.
How does deferred shading work?
Everything is rendered once, but it's done in a rather clever way. When rendering, a pixel shader is used to write all the data needed to different buffers. One for depth/normals, one for lighting (specular info, etc), and one for color. Then at the end a shader is used to combine all the information from the different buffers into a final picture.
This reduces rendering to once, but puts more stress on the graphics card and uses more video memory. This was not necessarily considered the better choice until recent years when graphics hardware has gotten better.
Why make the switch?
I hate to say "because the other guys are doing it" but if you look into it the big players that were using deferred lighting have made the change to deferred shading ( see: UE4 ). This will allow us to easily implement different lighting models like cook-torrance.
It also has the advantage of allowing some additional shaders and postfx that we couldn't really do before. Take for instance Lukas' SSGI shader. One big issue he ran into is the lack of an available albedo (or color) buffer to read just color data from. This is available with deferred shading.
What's the status?
We've been working on the conversion for about a week now and everything seems to be working quite well. There's a few missing features, and a handful of broken things but it's functional and working. Performance is about the same as before, which we consider to be a good thing since we haven't attempted any kind of optimization yet.
Conclusion
It's a pretty big change and will have a huge impact on materials, material shaders, and lighting. While we plan to continue working on this regardless for personal use, we would like to know if this is something the community is interested seeing make it's way into the main repo. This wouldn't be a 3.6 or 3.7 change, but further down the line (perhaps 4.0?).
Links
Link to the development repo:
github.com/andr3wmac/Torque3D/tree/deferred_shading
A guy's benchmarking of deferred lighting vs shading:
frictionalgames.blogspot.ca/2010/10/pre-pass-lighting-redux.html
An article that evaluates Deferred Lighting VS Deferred Shading:
gameangst.com/?p=141
How does deferred lighting work?
Everything is rendered twice. Once to calculate lighting, and then everything is rendered again + the previously obtained lighting information for your final lit scene. This is why a 30k poly model loaded into Torque will show 60k.
This may sound very expensive but it's made in a way that the light prepass is as cheap as possible. It's by no means a bad method of doing things, nor was it a poor choice at the time. In fact, most engines around that time frame used deferred lighting. It also has the advantage of running better on older hardware.
How does deferred shading work?
Everything is rendered once, but it's done in a rather clever way. When rendering, a pixel shader is used to write all the data needed to different buffers. One for depth/normals, one for lighting (specular info, etc), and one for color. Then at the end a shader is used to combine all the information from the different buffers into a final picture.
This reduces rendering to once, but puts more stress on the graphics card and uses more video memory. This was not necessarily considered the better choice until recent years when graphics hardware has gotten better.
Why make the switch?
I hate to say "because the other guys are doing it" but if you look into it the big players that were using deferred lighting have made the change to deferred shading ( see: UE4 ). This will allow us to easily implement different lighting models like cook-torrance.
It also has the advantage of allowing some additional shaders and postfx that we couldn't really do before. Take for instance Lukas' SSGI shader. One big issue he ran into is the lack of an available albedo (or color) buffer to read just color data from. This is available with deferred shading.
What's the status?
We've been working on the conversion for about a week now and everything seems to be working quite well. There's a few missing features, and a handful of broken things but it's functional and working. Performance is about the same as before, which we consider to be a good thing since we haven't attempted any kind of optimization yet.
Conclusion
It's a pretty big change and will have a huge impact on materials, material shaders, and lighting. While we plan to continue working on this regardless for personal use, we would like to know if this is something the community is interested seeing make it's way into the main repo. This wouldn't be a 3.6 or 3.7 change, but further down the line (perhaps 4.0?).
Links
Link to the development repo:
github.com/andr3wmac/Torque3D/tree/deferred_shading
A guy's benchmarking of deferred lighting vs shading:
frictionalgames.blogspot.ca/2010/10/pre-pass-lighting-redux.html
An article that evaluates Deferred Lighting VS Deferred Shading:
gameangst.com/?p=141
About the author
#82
Super-cool work you guys are doing here, following the progress with a lot of excitement.
07/13/2014 (4:17 pm)
Alright, glad to hear!Super-cool work you guys are doing here, following the progress with a lot of excitement.
#84
Thanks for the assist Haladrin.
08/13/2014 (8:09 pm)
I'll just leave this here: i.imgur.com/uJwNSRV.pngThanks for the assist Haladrin.
#85
* Glow should be reverted because while the system I was playing with is theoretically faster, it doesn't have varying levels of glow. It can only glow on and glow off, there isn't 50% glow or anything. I overlooked that and it's not worth packing glow mask into the gbuffer when we could just render via glowbin and call it a day. It was working before I "made it better" so it's just a revert.
08/14/2014 (1:53 pm)
Awesome. Dynamic cubemaps are off the list. That leaves:- Revert glow back to old system *
- Get the last of the terrain layers working
- Fix scattersky
- Review, comment and clean code + shaders.
* Glow should be reverted because while the system I was playing with is theoretically faster, it doesn't have varying levels of glow. It can only glow on and glow off, there isn't 50% glow or anything. I overlooked that and it's not worth packing glow mask into the gbuffer when we could just render via glowbin and call it a day. It was working before I "made it better" so it's just a revert.
#86
2) github.com/GarageGames/Torque3D/pull/710 with the noted adjustments sorted that most of the way. We will need to do one last pass to clean that on up. (Still get stippling for the 4th and 5th textures)
3) working. it's a hackjob, but it does show for normal and reflections now.
4) TODO
5) ???
6) profit
08/15/2014 (11:18 pm)
1) reverted.2) github.com/GarageGames/Torque3D/pull/710 with the noted adjustments sorted that most of the way. We will need to do one last pass to clean that on up. (Still get stippling for the 4th and 5th textures)
3) working. it's a hackjob, but it does show for normal and reflections now.
4) TODO
5) ???
6) profit
#87
08/21/2014 (8:33 pm)
github.com/Azaezel/Torque3D/tree/Deferred_Shading2 clean branch compatible with the latest dev-head. believe we've sorted most things on out. though would of course highly appreciate folks piling on and trying to break things. 
#88
08/21/2014 (8:50 pm)
Any additional build dependencies or anything? 
#91
08/22/2014 (2:47 am)
U make me cry :)
#92
08/22/2014 (8:44 am)
Is it already working with the OpenGL renderer (and the Linux port)?
#93
08/22/2014 (12:00 pm)
Luis asked me to wait a week or so on the port there, so not yet, but definitely still planning on it.
#94
i.imgur.com/Eun0yaq.jpg
Current
imgur.com/k0H89iV,BF06SuW two shots of the latest using the old testbed levels and assets. (Should note: The darkness in the second shot is due to the options panel gui background.)
This one contains a bit of a corner-case in terms of whether folks would consider it a fix or augmentation in the form of correcting gamma linearly. See filmicgames.com/archives/299 for a basic concept writeup, and www.youtube.com/watch?feature=player_detailpage&v=FQMbxzTUuSg#t=2282 for future use intent. The end result is that displayed textures are far closer to what-you-see-is-what-you-get when comparing color on the texturesheet to color in game. Because the prior gamma correction was also doubling as brightness, that and contrast were added as additional options.
While that obviously has an impact on present games, we did feel that the hassle of conversion outweighed the future hoops we'd be looking at jumping through to maintain essentially flawed math down the road.
08/27/2014 (5:00 pm)
Priori.imgur.com/Eun0yaq.jpg
Current
imgur.com/k0H89iV,BF06SuW two shots of the latest using the old testbed levels and assets. (Should note: The darkness in the second shot is due to the options panel gui background.)
This one contains a bit of a corner-case in terms of whether folks would consider it a fix or augmentation in the form of correcting gamma linearly. See filmicgames.com/archives/299 for a basic concept writeup, and www.youtube.com/watch?feature=player_detailpage&v=FQMbxzTUuSg#t=2282 for future use intent. The end result is that displayed textures are far closer to what-you-see-is-what-you-get when comparing color on the texturesheet to color in game. Because the prior gamma correction was also doubling as brightness, that and contrast were added as additional options.
While that obviously has an impact on present games, we did feel that the hassle of conversion outweighed the future hoops we'd be looking at jumping through to maintain essentially flawed math down the road.
#95
This is the texture used on that landscape:
And here's a shot of the same landscape and texture with gamma decoding:
As you can see the color is now much closer to the source image.
08/27/2014 (5:56 pm)
Here's a shot of the empty landscape from the level previews:This is the texture used on that landscape:
And here's a shot of the same landscape and texture with gamma decoding:
As you can see the color is now much closer to the source image.
#97
08/28/2014 (4:57 am)
fantastic..great work guys!!!
#99
08/30/2014 (8:49 am)
The linear space lighting is nice and easy to integrate. We integrated it into DHMC and it works great. it really cleans up the textures. Thanks guys for the hard work on figuring this stuff out.
#100
09/02/2014 (11:07 am)
I got deferred shading running in DHMC - www.playmech.com/images/deferredShading.png 
Andrew Mac
Please don't turn this into the PBR thread. That thread is a mess of links, screenshots, and pasted information that drowns out any attempts at a real technical discussion. It's information overload.
@ Felix
Spec-power and spec strength are in there now, and roughness will be taking the place of spec-power in the buffer when it's used. As for velocity, we have no plans for that now but it's the kind of thing that could be tacked on later so once the dust has settled we can explore that.