Physically Based Rendering
by Pierre DragoFire Hay · in Torque 3D Professional · 03/05/2014 (4:39 pm) · 156 replies
I've been slowly reading up on this and wonder if anyone else has looked into this or done any work with it and T3D yet?
For those who don't know what Physically Base Rendering or as some call it Physically Based Shading is here's a topic over at RSI covering what is physically based rendering, this covers most aspects of PBR without getting to technical.
What is PBR or sometimes refered to as PBS or BRDF
The most trivial explanation of a PBR/PBS/BRDF (physical based renderer / physical based shader / bidirectional reflectance distribution function) is that it is the bit of shader code describing how a surface reacts to light. Generally, it is responsible for calculating the specular highlights and diffuse characteristics of the surface material. They are mathematical approximations of how surfaces react to light in the real world. In computer graphics, we try to model the physical world as accurately as possible, but we are constrained by computation. For this reason, the mathematical efficiency of BRDFs is very important. Some of the better known BRDFs - Blinn, Phong and Lambert, for instance - are well known for this reason: they are computationally inexpensive to calculate and intuitive to adjust. However they compromise efficiency for accuracy. If we concern ourselves with more expensive and more accurate models, we uncover a second layer of shading models: Oren-Nayar, Cook-Torrance, Askikhmin-Shirley, etc.
There is no single model that fits every situation, but there are some better than others. The Cook-Torrance model has been shown to be a top performer, when compared against actual acquired BRDF data. Of course, with the good comes the bad and Cook-Torrance is one of the most expensive models to compute. But for overall results, it is hard to beat. So this is our target; a nice implementation of the Cook-Torrance reflectance model. Now this maybe an issue when using Cook-Torrance model on mobiles and consoles due to hardware limitations, so it's a question of which module to use.
What's required?
Base PBR implementation consists of 3 things;
Gamma-correct rendering
[ul]Realistic rendering requires handling values much higher than display white (1.0) Before shading: light intensities, lightmaps, environment maps Shading produces highlights that affect bloom, fog, DoF, motion blur, etc. Cheap solutions exist Good tone mapping (ideally filmic)
Difference between textures and materials.
There are few fundamental distinctions between textures and materials. For example, you cannot apply a texture on a static mesh or BSP geometry. Textures have to be a part of a material. The material is what you would use to texture your environment and apply to Static Meshes.
Here are the differences between a texture and a material:
Textures is a single file, a 2d static image. It is usually a diffuse, specular or a normal map file that you would create in Photoshop or Gimp, as a tga, tiff, bmp, png file. These can be manipulated photographs, hand-painted textures or textures baked in an application such as xNormals.
Materials are made up of various textures combined together inside a Material Editor(in-engine or 3rd party editor). Materials include various textures and material expressions that creates a network of nodes. The final result is a material you can use to apply on your BSP geometry and on Static Meshes. Materials are what you see rendered in-game.
It should be noted that Specular[i] map on a texture, is connected to the [i]refractive index and as such describes a physical property. The shading model then varies this reflectance based on view angle and surface roughness. Thus the surface roughness is adjusted to create variety and specular not be varied for a given material.
A video on PBR for Artist
PBR in Substance by Allegorithmic
Here's a few examples of PBR in action.
tri-Ace Technical Demo Trailer 2011 "Physically-based Rendering"
Star Citizen Avenger PBR
Useful Links:
Pixar writing a Cook-Torrance surface Shader
Pixar writing a BRDF template
Pixar writing a BRDF template part2
Pixar CookTorrance Slim Template
Shading course 2012
Shading course 2013
Physically Based Lighting in Black-Ops
Shader code for physically based lighting
Basic Theory of Physically-Based Rendering
Paprika Render
Mitsuba Render
PBRT Org.
Unreal PBR
Houdini Shaders for physically based rendering (PBR)
D3DBook:(Lighting) Cook-Torrance
Specular BRDF Reference by Brian Karis from Epic game
Books of Interest:
Physically Based Rendering, From Theory to Implementation
ShaderX7
The RenderMan, Shading language Guide
Edit: Updated with new information and links.
For those who don't know what Physically Base Rendering or as some call it Physically Based Shading is here's a topic over at RSI covering what is physically based rendering, this covers most aspects of PBR without getting to technical.
What is PBR or sometimes refered to as PBS or BRDF
The most trivial explanation of a PBR/PBS/BRDF (physical based renderer / physical based shader / bidirectional reflectance distribution function) is that it is the bit of shader code describing how a surface reacts to light. Generally, it is responsible for calculating the specular highlights and diffuse characteristics of the surface material. They are mathematical approximations of how surfaces react to light in the real world. In computer graphics, we try to model the physical world as accurately as possible, but we are constrained by computation. For this reason, the mathematical efficiency of BRDFs is very important. Some of the better known BRDFs - Blinn, Phong and Lambert, for instance - are well known for this reason: they are computationally inexpensive to calculate and intuitive to adjust. However they compromise efficiency for accuracy. If we concern ourselves with more expensive and more accurate models, we uncover a second layer of shading models: Oren-Nayar, Cook-Torrance, Askikhmin-Shirley, etc.
There is no single model that fits every situation, but there are some better than others. The Cook-Torrance model has been shown to be a top performer, when compared against actual acquired BRDF data. Of course, with the good comes the bad and Cook-Torrance is one of the most expensive models to compute. But for overall results, it is hard to beat. So this is our target; a nice implementation of the Cook-Torrance reflectance model. Now this maybe an issue when using Cook-Torrance model on mobiles and consoles due to hardware limitations, so it's a question of which module to use.
What's required?
Base PBR implementation consists of 3 things;
Gamma-correct rendering
- Shading inputs (textures, light colors, vertex colors, etc.) naturally authored, previewed [li]and (often) stored with nonlinear (gamma) encoding
- Final frame buffer also uses nonlinear encoding
- This is done for good reasons
- [li]Perceptually uniform(ish) = efficient use of bits
- Legacy reasons (tools, file formats, hardware)
[ul]
Difference between textures and materials.
There are few fundamental distinctions between textures and materials. For example, you cannot apply a texture on a static mesh or BSP geometry. Textures have to be a part of a material. The material is what you would use to texture your environment and apply to Static Meshes.
Here are the differences between a texture and a material:
Textures is a single file, a 2d static image. It is usually a diffuse, specular or a normal map file that you would create in Photoshop or Gimp, as a tga, tiff, bmp, png file. These can be manipulated photographs, hand-painted textures or textures baked in an application such as xNormals.
Materials are made up of various textures combined together inside a Material Editor(in-engine or 3rd party editor). Materials include various textures and material expressions that creates a network of nodes. The final result is a material you can use to apply on your BSP geometry and on Static Meshes. Materials are what you see rendered in-game.
It should be noted that Specular[i] map on a texture, is connected to the [i]refractive index and as such describes a physical property. The shading model then varies this reflectance based on view angle and surface roughness. Thus the surface roughness is adjusted to create variety and specular not be varied for a given material.
A video on PBR for Artist
PBR in Substance by Allegorithmic
Here's a few examples of PBR in action.
tri-Ace Technical Demo Trailer 2011 "Physically-based Rendering"
Star Citizen Avenger PBR
Useful Links:
Pixar writing a Cook-Torrance surface Shader
Pixar writing a BRDF template
Pixar writing a BRDF template part2
Pixar CookTorrance Slim Template
Shading course 2012
Shading course 2013
Physically Based Lighting in Black-Ops
Shader code for physically based lighting
Basic Theory of Physically-Based Rendering
Paprika Render
Mitsuba Render
PBRT Org.
Unreal PBR
Houdini Shaders for physically based rendering (PBR)
D3DBook:(Lighting) Cook-Torrance
Specular BRDF Reference by Brian Karis from Epic game
Books of Interest:
Physically Based Rendering, From Theory to Implementation
ShaderX7
The RenderMan, Shading language Guide
Edit: Updated with new information and links.
#22
If you want that the thread will not get lost in the forum and the T3D committee may have this for future decisions, I recommend:
03/17/2014 (1:51 pm)
A very interesting topic. If you want that the thread will not get lost in the forum and the T3D committee may have this for future decisions, I recommend:
- Recopile all important information in header post.
- Add a issue on github with PBR feature request
- We have a vote system for feature request. PBR entry
#23
PBR was added to the feature request list and has quite a few votes, unless the new committee has started a new feature request page..but anyways I found this: www.luxrender.net/en_GB/index src.luxrender.net/lux/src
Not for sure if it's any help but looks like it may be a step in the right direction..
03/17/2014 (2:17 pm)
@LuisPBR was added to the feature request list and has quite a few votes, unless the new committee has started a new feature request page..but anyways I found this: www.luxrender.net/en_GB/index src.luxrender.net/lux/src
Not for sure if it's any help but looks like it may be a step in the right direction..
#24
General introduction
See Marmoset (Team who build the incredible PBR shaders for Unity)
http://www.marmoset.co/toolbag/learn/pbr-theory
more background:
https://developer.nvidia.com/sites/default/files/akamai/gamedev/files/gdc12/GDC2012_Mastering_DirectX11_with_Unity.pdf
http://www.unrealengine.com/files/downloads/2013SiggraphPresentationsNotes.pdf
For implementation,
Kostas Anagnostou has probably the best collection of links on implementing , underlying theory and other examples. His site is worth spending a chunk of time reading through. see:
http://interplayoflight.wordpress.com/2013/12/30/readings-on-physically-based-rendering/
Sebastien Lagarde's blog posts are also worth a close read
http://seblagarde.wordpress.com/2011/08/17/feeding-a-physical-based-lighting-mode/
http://seblagarde.wordpress.com/2011/08/17/hello-world/
Siggraph links (always worth a read)
http://blog.selfshadow.com/publications/s2013-shading-course/
Hope this helps.
p.
03/17/2014 (8:48 pm)
In case its useful (and dispels some confusion) the following links might help give some background and are worth reading for those who want to experiment:General introduction
See Marmoset (Team who build the incredible PBR shaders for Unity)
http://www.marmoset.co/toolbag/learn/pbr-theory
more background:
https://developer.nvidia.com/sites/default/files/akamai/gamedev/files/gdc12/GDC2012_Mastering_DirectX11_with_Unity.pdf
http://www.unrealengine.com/files/downloads/2013SiggraphPresentationsNotes.pdf
For implementation,
Kostas Anagnostou has probably the best collection of links on implementing , underlying theory and other examples. His site is worth spending a chunk of time reading through. see:
http://interplayoflight.wordpress.com/2013/12/30/readings-on-physically-based-rendering/
Sebastien Lagarde's blog posts are also worth a close read
http://seblagarde.wordpress.com/2011/08/17/feeding-a-physical-based-lighting-mode/
http://seblagarde.wordpress.com/2011/08/17/hello-world/
Siggraph links (always worth a read)
http://blog.selfshadow.com/publications/s2013-shading-course/
Hope this helps.
p.
#25
03/21/2014 (11:56 am)
I don't think that the rendering is a problem at this point now. We are already implementing OpenGL and DirectX 11. It shouldn't be a problem, just a lot of work and effort from a experienced developer. Although, this is the kind of thing that Torque3D needs. Sub-division surface scattering, Godrays, etc. If anyone were to implement this it would never be a waste of time.
#26
03/21/2014 (2:00 pm)
I totally agree
#27
03/21/2014 (8:46 pm)
It will involve a large amount of research and development. Just reading through a small amount if the information I have managed to locate and also purchase I can see the possibilities of this and roughly how much work would be required to make it work with T3D.
#28
eg.
Raw state (natural state without finish or machining)
Basic state (basic machining)
Machined state (planed and machined)
Refined state (sanded to smooth finish)
Polished state (fine sanded and highly polished)
Prime state (lacquered finish)
Ideally other stated would include damaged, scratched, rusted, pitted, etc... and these would be for each of the above states for each material. So as you can see there is a lot of research required not just for the core PBR coding, but all the materials also.
03/24/2014 (6:40 pm)
From what I've read the real issue is the material library. Now I've managed to get some basic material information, but to be truly effective the library required as many as 5 versions for every material to allow for the different states it maybe in, and ideally more that 5 would be better.eg.
Raw state (natural state without finish or machining)
Basic state (basic machining)
Machined state (planed and machined)
Refined state (sanded to smooth finish)
Polished state (fine sanded and highly polished)
Prime state (lacquered finish)
Ideally other stated would include damaged, scratched, rusted, pitted, etc... and these would be for each of the above states for each material. So as you can see there is a lot of research required not just for the core PBR coding, but all the materials also.
#29
03/25/2014 (4:31 am)
I see..sounds like you are understanding it and making a little progress.
#31
Here is the website:
http://paprikarender.github.io/
Here is also another renderer:
http://www.mitsuba-renderer.org/
Good Luck!
Another note is, well... to be honest, if you need some time of renderer or physics system or particle system, you could probably find an open-source project of it. Their not always the best but its usually around waiting for someone to use it. I'm not saying this is the best, but forgive my language...
ITS PRETTY DANG GOOD.
PS: Try the demos, they look pretty sweet!
04/03/2014 (2:05 pm)
Bringing this topic back to life, I just found a open-source Physically Based Renderer based on partly PBRT and partly (good old) NVIDIA Gelato. Its called Paprika Render and depends on the OSL language. This can run on Linux, ultimately meaning its cross-platform. If anyone wants to use this resource for their own game, I encourage you to do so. And if Pierre wants to integrate it into the Material Editor that would be awesome too. I just don't have enough time or experience whatsoever to do so.Here is the website:
http://paprikarender.github.io/
Here is also another renderer:
http://www.mitsuba-renderer.org/
Good Luck!
Another note is, well... to be honest, if you need some time of renderer or physics system or particle system, you could probably find an open-source project of it. Their not always the best but its usually around waiting for someone to use it. I'm not saying this is the best, but forgive my language...
ITS PRETTY DANG GOOD.
PS: Try the demos, they look pretty sweet!
#32
04/04/2014 (1:39 am)
thanks Raa. 
#33
From <Haladrin> when discussing applicability of repurposing the present material feature structure :
cook-torrance
www.pbrt.org/ and github.com/mmp/pbrt-v2 BSD Licence.
05/14/2014 (12:47 pm)
A few additional notes to add:From <Haladrin> when discussing applicability of repurposing the present material feature structure :
cook-torrance
//
// Microfacet Specular Cook-Torrance
//
// D( h ) F( v, h ) G( l, v, h )
// f( l, v ) = ____________________________
// 4 ( dot( n, l ) dot( n, v )
//
//
//
// D( h ) ==> Normal distribution function
//
//
// or GGX ( disney / unreal 4 )
//
// alpha = pow( roughness, 2 );
//
// pow( alpha, 2 )
// D( h ) = ________________________________________________________________
// PI pow( pow( nDotH , 2 ) ( pow( alpha, 2 ) - 1 ) + 1 ), 2 )
//
//
// G( l, v, h ) ==> Geometric attenution, basicly a visibility term
//
//
// G( L, V, h ) = G( L ) G( V )
//
// nDotL
// G( L ) = _________________________
// nDotL ( 1 - k ) + k
//
//
// nDotV
// G( V ) = _________________________
// nDotV ( 1 - k ) + k
//
//
// pow( ( Roughness + 1 ), 2)
// , Where k = __________________________ ( unreal 4 )
// 8
//
//
// F( v, h ) ==> frensel term, slicks approach.
//
//
// F( v, h ) = F0 + ( 1.0 - F0 ) * pow( 1.0f - HDotV, 5.0f )
//
//
// where
//
// F0 = BaseColor * nDotL
//
// Dielectric materials always have a range of 0.02 < F0 < 0.05 , use a stock value of 0.04 ( roughly plastics )
//
// Fc = pow( 1.0f - HDotV, 5.0f );
// Fdielectric = 0.04 + ( 1.0 - 0.04 ) * Fc;
// Fconductor = surface.baseColor + ( float3( 1.0, 1.0, 1.0 ) - surface.baseColor ) * Fc;www.pbrt.org/ and github.com/mmp/pbrt-v2 BSD Licence.
#34
Just my two cents.
05/14/2014 (2:09 pm)
I personally don't know enough to implement Physically Based Rendering, but if someone does implement it (not asking anyone to), Torque 3D would be able to compete with Unreal Engine 4. PBR is that big of a deal, and Torque3D would gain a huge upgrade from that system.Just my two cents.
#35
DirectX 11 adds features and enhancements for the hardware compatibility issues that are on the market. PBR/PBS is coded to tell DirectX/OpenGL, etc... how to render the model. So using DX11 with PBR/PBS would add so much more to Torque3D.
@raa,
I'm surprised no-one else thought about adding PBR (or PBS for some) earlier, it's been around since the early 2000's.
05/19/2014 (3:28 pm)
@andrew,DirectX 11 adds features and enhancements for the hardware compatibility issues that are on the market. PBR/PBS is coded to tell DirectX/OpenGL, etc... how to render the model. So using DX11 with PBR/PBS would add so much more to Torque3D.
@raa,
I'm surprised no-one else thought about adding PBR (or PBS for some) earlier, it's been around since the early 2000's.
#36
06/05/2014 (9:55 pm)
This came up the other day from a couple different directions as well: For those familiar with the requisite math, but unfamiliar with the syntax the shadergen system uses to generate hlsl/glsl files, adding gameclay.com/documentation/t3d-materials to the list of datapoints. Still relatively accurate from what I've been able to tell thus-far, though in my case it's been hijacking existing material shader writes and inserting game-relevant tweaks. (I suspect it would take additional alterations to the advanced lighting code as well.)
#37
Albedo would correspond to diffuse (github.com/GarageGames/Torque3D/blob/development/Engine/source/shaderGen/HLSL/sh...)
Specular/Metallic would correspond to specular,
Roughness/Glossiness actually already exists in-engine in the form of an alpha channel in the specular texture. that actually came up recently in a PR dealing with the
github.com/GarageGames/Torque3D/blob/development/Engine/source/lighting/advanced... method.
and normal as bump. (github.com/GarageGames/Torque3D/blob/development/Engine/source/shaderGen/HLSL/bu... )
06/06/2014 (3:15 pm)
Squeezing in a bit more research time. Going off of the list of textures: www.artisaverb.info/PBT.htmlAlbedo would correspond to diffuse (github.com/GarageGames/Torque3D/blob/development/Engine/source/shaderGen/HLSL/sh...)
Specular/Metallic would correspond to specular,
Roughness/Glossiness actually already exists in-engine in the form of an alpha channel in the specular texture. that actually came up recently in a PR dealing with the
github.com/GarageGames/Torque3D/blob/development/Engine/source/lighting/advanced... method.
and normal as bump. (github.com/GarageGames/Torque3D/blob/development/Engine/source/shaderGen/HLSL/bu... )
#38
As I understood it, pureLIGHT was added to T3D sometime ago (this might be true or not).
But have wondered about the use of the SSAO Kit and pureLIGHT combined to improve the shader side of things, working to the higher end of shaders that are within PBR/PBS.
06/06/2014 (6:39 pm)
I thought there was something in T3D already, and it might just need some fine tuning. But could find very little on the topic.As I understood it, pureLIGHT was added to T3D sometime ago (this might be true or not).
But have wondered about the use of the SSAO Kit and pureLIGHT combined to improve the shader side of things, working to the higher end of shaders that are within PBR/PBS.
#39
Purelight isn't rolled in, no, though there do seem to be remnants of the capacity to bake lighting still lurking around if advanced lighting is shut off entirely and it goes into fixed-function mode. (Won't even let you try to generate mission lighting otherwise.)
The rest of that stuff I've been jotting down thusfar has been collating injection points for relevant code alterations when it comes to the system that generates those hlsls that'll end up in game\shaders\procedural. (glsl files as well if using the beamng opengl port Luis is spearheading)
06/07/2014 (12:38 am)
SSAO comes stock with the engine. You can find that under the edit->"postEffect manager" in the level editor. Purelight isn't rolled in, no, though there do seem to be remnants of the capacity to bake lighting still lurking around if advanced lighting is shut off entirely and it goes into fixed-function mode. (Won't even let you try to generate mission lighting otherwise.)
The rest of that stuff I've been jotting down thusfar has been collating injection points for relevant code alterations when it comes to the system that generates those hlsls that'll end up in game\shaders\procedural. (glsl files as well if using the beamng opengl port Luis is spearheading)
#40
I haven't touched the shader gen stuff in T3D before but i'll give it a go in the coming weeks.
I'll fire an email to andrew in a few weeks and work out some details because there are a few issues i can already think about with implementing it.
06/07/2014 (1:38 am)
I implemented PBS recently for a model viewer i develop for a game.I haven't touched the shader gen stuff in T3D before but i'll give it a go in the coming weeks.
I'll fire an email to andrew in a few weeks and work out some details because there are a few issues i can already think about with implementing it.
Andrew Mac
My question is, would it be possible to achieve both at the same time? In other words, could I have an object with Diffuse/Normal/Spec and regular advanced lighting while still using PBR on a different object? I'm just trying to map out the steps needed to make this a reality. I'm just wondering if the idea here is to gut torque's material system or extend it to support PBR.
Either way, I'm in for whatever help I can provide.