Physically Based Rendering
by Pierre DragoFire Hay · in Torque 3D Professional · 03/05/2014 (4:39 pm) · 156 replies
I've been slowly reading up on this and wonder if anyone else has looked into this or done any work with it and T3D yet?
For those who don't know what Physically Base Rendering or as some call it Physically Based Shading is here's a topic over at RSI covering what is physically based rendering, this covers most aspects of PBR without getting to technical.
What is PBR or sometimes refered to as PBS or BRDF
The most trivial explanation of a PBR/PBS/BRDF (physical based renderer / physical based shader / bidirectional reflectance distribution function) is that it is the bit of shader code describing how a surface reacts to light. Generally, it is responsible for calculating the specular highlights and diffuse characteristics of the surface material. They are mathematical approximations of how surfaces react to light in the real world. In computer graphics, we try to model the physical world as accurately as possible, but we are constrained by computation. For this reason, the mathematical efficiency of BRDFs is very important. Some of the better known BRDFs - Blinn, Phong and Lambert, for instance - are well known for this reason: they are computationally inexpensive to calculate and intuitive to adjust. However they compromise efficiency for accuracy. If we concern ourselves with more expensive and more accurate models, we uncover a second layer of shading models: Oren-Nayar, Cook-Torrance, Askikhmin-Shirley, etc.
There is no single model that fits every situation, but there are some better than others. The Cook-Torrance model has been shown to be a top performer, when compared against actual acquired BRDF data. Of course, with the good comes the bad and Cook-Torrance is one of the most expensive models to compute. But for overall results, it is hard to beat. So this is our target; a nice implementation of the Cook-Torrance reflectance model. Now this maybe an issue when using Cook-Torrance model on mobiles and consoles due to hardware limitations, so it's a question of which module to use.
What's required?
Base PBR implementation consists of 3 things;
Gamma-correct rendering
[ul]Realistic rendering requires handling values much higher than display white (1.0) Before shading: light intensities, lightmaps, environment maps Shading produces highlights that affect bloom, fog, DoF, motion blur, etc. Cheap solutions exist Good tone mapping (ideally filmic)
Difference between textures and materials.
There are few fundamental distinctions between textures and materials. For example, you cannot apply a texture on a static mesh or BSP geometry. Textures have to be a part of a material. The material is what you would use to texture your environment and apply to Static Meshes.
Here are the differences between a texture and a material:
Textures is a single file, a 2d static image. It is usually a diffuse, specular or a normal map file that you would create in Photoshop or Gimp, as a tga, tiff, bmp, png file. These can be manipulated photographs, hand-painted textures or textures baked in an application such as xNormals.
Materials are made up of various textures combined together inside a Material Editor(in-engine or 3rd party editor). Materials include various textures and material expressions that creates a network of nodes. The final result is a material you can use to apply on your BSP geometry and on Static Meshes. Materials are what you see rendered in-game.
It should be noted that Specular[i] map on a texture, is connected to the [i]refractive index and as such describes a physical property. The shading model then varies this reflectance based on view angle and surface roughness. Thus the surface roughness is adjusted to create variety and specular not be varied for a given material.
A video on PBR for Artist
PBR in Substance by Allegorithmic
Here's a few examples of PBR in action.
tri-Ace Technical Demo Trailer 2011 "Physically-based Rendering"
Star Citizen Avenger PBR
Useful Links:
Pixar writing a Cook-Torrance surface Shader
Pixar writing a BRDF template
Pixar writing a BRDF template part2
Pixar CookTorrance Slim Template
Shading course 2012
Shading course 2013
Physically Based Lighting in Black-Ops
Shader code for physically based lighting
Basic Theory of Physically-Based Rendering
Paprika Render
Mitsuba Render
PBRT Org.
Unreal PBR
Houdini Shaders for physically based rendering (PBR)
D3DBook:(Lighting) Cook-Torrance
Specular BRDF Reference by Brian Karis from Epic game
Books of Interest:
Physically Based Rendering, From Theory to Implementation
ShaderX7
The RenderMan, Shading language Guide
Edit: Updated with new information and links.
For those who don't know what Physically Base Rendering or as some call it Physically Based Shading is here's a topic over at RSI covering what is physically based rendering, this covers most aspects of PBR without getting to technical.
What is PBR or sometimes refered to as PBS or BRDF
The most trivial explanation of a PBR/PBS/BRDF (physical based renderer / physical based shader / bidirectional reflectance distribution function) is that it is the bit of shader code describing how a surface reacts to light. Generally, it is responsible for calculating the specular highlights and diffuse characteristics of the surface material. They are mathematical approximations of how surfaces react to light in the real world. In computer graphics, we try to model the physical world as accurately as possible, but we are constrained by computation. For this reason, the mathematical efficiency of BRDFs is very important. Some of the better known BRDFs - Blinn, Phong and Lambert, for instance - are well known for this reason: they are computationally inexpensive to calculate and intuitive to adjust. However they compromise efficiency for accuracy. If we concern ourselves with more expensive and more accurate models, we uncover a second layer of shading models: Oren-Nayar, Cook-Torrance, Askikhmin-Shirley, etc.
There is no single model that fits every situation, but there are some better than others. The Cook-Torrance model has been shown to be a top performer, when compared against actual acquired BRDF data. Of course, with the good comes the bad and Cook-Torrance is one of the most expensive models to compute. But for overall results, it is hard to beat. So this is our target; a nice implementation of the Cook-Torrance reflectance model. Now this maybe an issue when using Cook-Torrance model on mobiles and consoles due to hardware limitations, so it's a question of which module to use.
What's required?
Base PBR implementation consists of 3 things;
Gamma-correct rendering
- Shading inputs (textures, light colors, vertex colors, etc.) naturally authored, previewed [li]and (often) stored with nonlinear (gamma) encoding
- Final frame buffer also uses nonlinear encoding
- This is done for good reasons
- [li]Perceptually uniform(ish) = efficient use of bits
- Legacy reasons (tools, file formats, hardware)
[ul]
Difference between textures and materials.
There are few fundamental distinctions between textures and materials. For example, you cannot apply a texture on a static mesh or BSP geometry. Textures have to be a part of a material. The material is what you would use to texture your environment and apply to Static Meshes.
Here are the differences between a texture and a material:
Textures is a single file, a 2d static image. It is usually a diffuse, specular or a normal map file that you would create in Photoshop or Gimp, as a tga, tiff, bmp, png file. These can be manipulated photographs, hand-painted textures or textures baked in an application such as xNormals.
Materials are made up of various textures combined together inside a Material Editor(in-engine or 3rd party editor). Materials include various textures and material expressions that creates a network of nodes. The final result is a material you can use to apply on your BSP geometry and on Static Meshes. Materials are what you see rendered in-game.
It should be noted that Specular[i] map on a texture, is connected to the [i]refractive index and as such describes a physical property. The shading model then varies this reflectance based on view angle and surface roughness. Thus the surface roughness is adjusted to create variety and specular not be varied for a given material.
A video on PBR for Artist
PBR in Substance by Allegorithmic
Here's a few examples of PBR in action.
tri-Ace Technical Demo Trailer 2011 "Physically-based Rendering"
Star Citizen Avenger PBR
Useful Links:
Pixar writing a Cook-Torrance surface Shader
Pixar writing a BRDF template
Pixar writing a BRDF template part2
Pixar CookTorrance Slim Template
Shading course 2012
Shading course 2013
Physically Based Lighting in Black-Ops
Shader code for physically based lighting
Basic Theory of Physically-Based Rendering
Paprika Render
Mitsuba Render
PBRT Org.
Unreal PBR
Houdini Shaders for physically based rendering (PBR)
D3DBook:(Lighting) Cook-Torrance
Specular BRDF Reference by Brian Karis from Epic game
Books of Interest:
Physically Based Rendering, From Theory to Implementation
ShaderX7
The RenderMan, Shading language Guide
Edit: Updated with new information and links.
#62
Here is a great article from UE4. de45xmedrsdbp.cloudfront.net/Resources/files/2013SiggraphPresentationsNotes-2691...
One key quote is this one
06/07/2014 (7:36 pm)
The thing is none of the other AAA engines are implementing PBS to the detail that pierre is describing.Here is a great article from UE4. de45xmedrsdbp.cloudfront.net/Resources/files/2013SiggraphPresentationsNotes-2691...
One key quote is this one
Quote:Our material model is a simplification of Disneys, with an eye towards efficiency for real-time rendering.
Limiting the number of parameters is extremely important for optimizing G-Buffer space,
reducing texture storage and access, and minimizing the cost of blending material layers in the pixel
shader.
#63
Now that does mean folks'll still have to keep in mind the instruction limit (I assume that's still a thing, but I've been messing on and off with video programming since we were dealing with CRTs, so do sometimes lose track.), but by the same token, if it aint needed, it can be coded in such a manner as the datatracking is simply tossed out.
06/07/2014 (7:44 pm)
Quote:The thing is none of the other AAA engines are implementing PBS to the detail that pierre is describing.Well, that's actually one of the beauties of the shadergen system, and why I brought that one up over the custommaterial one. If you take a look up a bit in that last link I posted, you'll find a line
if( fd.features[ MFT_GlossMap ] )where it actively inserts and tracks a given variable on an as-defined basis.
Now that does mean folks'll still have to keep in mind the instruction limit (I assume that's still a thing, but I've been messing on and off with video programming since we were dealing with CRTs, so do sometimes lose track.), but by the same token, if it aint needed, it can be coded in such a manner as the datatracking is simply tossed out.
#64
Now I have been reading through Matt Pharr (principal Engineer at Intel) and Greg Humphreys (member of the NVIDIA OptiX raytracking team) book "PHYSICALLY BASED RENDERING, from theory to implementation" second edition, not the lightest book to read being some 1165 pages on maths, physics and coding, plus throw in "The RenderMan, Shading Language Guide", then "ShaderX7, Advanced Rendering Techniques", plus other books relating to the topic of rendering.
As you can guess there is a massive amount of information, data and views on how rendering can be done.
I'll add the following for those not 100% sure what Physically Based Rendering is.
It's a way to make everything look more photorealistic/natural-looking by changing how light bounces off everything. Since lighting controls how you see everything in the game (because you couldn't see in the pitch-black dark), it affects how every single part of the game looks.
Basically: instead of current lighting techniques like using multiple diffuse textures/specular maps for each part of every object in the game to represent different conditions, they can just create 1 texture for each part then artificially define properties like a refractive index to help parameterize a physics model that controls how light and shadow work when rendering frames of an in-game scene that contains that object.
In most implementations the physics model basically uses a predictive set of converging functions to determine how light from a specific source will reflect/refract off a given surface with different reflectivity and absorption/diffusion characteristics, which then refracts and reflects off other surfaces at a) different angles, with b) reduced intensity and c) a different wavelength, etc.
In productivity terms: people creating textures for in-game assets now have to spend less time creating multiple maps for each surface because they can just say "this panel is steel" or "this seat cushion is leather" with specific reflectivity/diffusion/texture/etc. rather than having to create multiple different copies of them that behave differently under different lighting conditions (such as in space, in atmosphere, indoors, etc.)
In visual terms: different types of surfaces (such as metal, leather, plastic, glass, etc.) should look more photorealistic and more "natural" because the way that light reflects off them and the way shadows are created will be more accurate.
A good artical is Basic Theory of Physically-Based Rendering
By Jeff Russell
Just a little tit-bit;
Gloss or roughness maps define the microsurface of the material (how rough or smooth it is), and do not replace a specular intensity map. However, if you are not used to working with gloss maps, it may be somewhat of an adjustment to put certain detail in the gloss map that you would otherwise add to the specular map.
Is specular no longer a thing?
Not quite. Specular reflection intensity, or reflectivity is still a very important parameter in PBR systems. You may not have a map to directly set reflectivity (e.g. with a metalness workflow) but it is still required in a PBR system.
06/07/2014 (8:39 pm)
I think the basic parameters are a must have. Now I have been reading through Matt Pharr (principal Engineer at Intel) and Greg Humphreys (member of the NVIDIA OptiX raytracking team) book "PHYSICALLY BASED RENDERING, from theory to implementation" second edition, not the lightest book to read being some 1165 pages on maths, physics and coding, plus throw in "The RenderMan, Shading Language Guide", then "ShaderX7, Advanced Rendering Techniques", plus other books relating to the topic of rendering.
As you can guess there is a massive amount of information, data and views on how rendering can be done.
I'll add the following for those not 100% sure what Physically Based Rendering is.
It's a way to make everything look more photorealistic/natural-looking by changing how light bounces off everything. Since lighting controls how you see everything in the game (because you couldn't see in the pitch-black dark), it affects how every single part of the game looks.
Basically: instead of current lighting techniques like using multiple diffuse textures/specular maps for each part of every object in the game to represent different conditions, they can just create 1 texture for each part then artificially define properties like a refractive index to help parameterize a physics model that controls how light and shadow work when rendering frames of an in-game scene that contains that object.
In most implementations the physics model basically uses a predictive set of converging functions to determine how light from a specific source will reflect/refract off a given surface with different reflectivity and absorption/diffusion characteristics, which then refracts and reflects off other surfaces at a) different angles, with b) reduced intensity and c) a different wavelength, etc.
In productivity terms: people creating textures for in-game assets now have to spend less time creating multiple maps for each surface because they can just say "this panel is steel" or "this seat cushion is leather" with specific reflectivity/diffusion/texture/etc. rather than having to create multiple different copies of them that behave differently under different lighting conditions (such as in space, in atmosphere, indoors, etc.)
In visual terms: different types of surfaces (such as metal, leather, plastic, glass, etc.) should look more photorealistic and more "natural" because the way that light reflects off them and the way shadows are created will be more accurate.
A good artical is Basic Theory of Physically-Based Rendering
By Jeff Russell
Just a little tit-bit;
Gloss or roughness maps define the microsurface of the material (how rough or smooth it is), and do not replace a specular intensity map. However, if you are not used to working with gloss maps, it may be somewhat of an adjustment to put certain detail in the gloss map that you would otherwise add to the specular map.
Is specular no longer a thing?
Not quite. Specular reflection intensity, or reflectivity is still a very important parameter in PBR systems. You may not have a map to directly set reflectivity (e.g. with a metalness workflow) but it is still required in a PBR system.
#65
06/07/2014 (8:53 pm)
Well regardless, we're going to be attempting to put in something similiar to the Unreal engine PBR/PBS. It's the best chance we've got at getting something that will actually be usable in games.
#66
any reason Unreal version? CryTEK already have one working, while as I understand it UDK are still developing it!
I ask as I've seen more for other development houses that have done more than Unreal has.
06/07/2014 (9:47 pm)
@Andrew any reason Unreal version? CryTEK already have one working, while as I understand it UDK are still developing it!
I ask as I've seen more for other development houses that have done more than Unreal has.
#67
UE4 screenshot showing roughness differences.
06/07/2014 (10:05 pm)
Don't confuse the old UDK with the new UE4. UE4 PBS system is working perfectly fine.UE4 screenshot showing roughness differences.
#68
06/07/2014 (10:07 pm)
seem every man and his dog are trying to get on the PBR wagon, even Unity is developing a PBS module.
#70
06/08/2014 (3:28 pm)
This is so cool, so is are you guys really going to work on PBR in T3D?
#72
seblagarde.wordpress.com/2014/04/14/dontnod-physically-based-rendering-chart-for...
seblagarde.wordpress.com/2012/04/30/dontnod-specular-and-glossiness-chart/
06/08/2014 (6:04 pm)
Some additional notes from the head artist as far as expected value-charts:seblagarde.wordpress.com/2014/04/14/dontnod-physically-based-rendering-chart-for...
seblagarde.wordpress.com/2012/04/30/dontnod-specular-and-glossiness-chart/
#73
that's the plan.
@Azaezel
Check out the Allegorithmic Substance Designer and Bitmap2Material programs, they allow for PBR material construction and already have a large number of materials already available.
The trick will be to integrate these with the PBR module we plan to add to T3D, it already has integrations into Unity, UE4, MAYA, 3DS MAX, MODO amd UDK/UE3
06/08/2014 (7:44 pm)
@raathat's the plan.
@Azaezel
Check out the Allegorithmic Substance Designer and Bitmap2Material programs, they allow for PBR material construction and already have a large number of materials already available.
The trick will be to integrate these with the PBR module we plan to add to T3D, it already has integrations into Unity, UE4, MAYA, 3DS MAX, MODO amd UDK/UE3
#74
06/08/2014 (8:03 pm)
Yeah, was just going over the marmoset toolkit on our company hangout with the head artist tonight as far as expectations set by the toolsets they work with. Also has a substance painter converter built on in.
#75
06/08/2014 (8:06 pm)
I agree Pierre, we should make our PBR implementation very similar to other engines. I also suggest that we handle our shaders with a visual shader creating. Someone made a visual shader system, so we should get it compatible with T3D's new PBR implementation.
#76
I don't know if either of these threads came to/near fruition. With Lucas, Rons, or Elfalcons work.
IBL:
www.garagegames.com/community/forums/viewthread/130483
SSDO:
www.garagegames.com/community/forums/viewthread/136081
06/09/2014 (2:25 pm)
Drew++ at polycount made this image for me to show you all why you're going to have to implement a IBL setup with possibly a SSDO addition as well at the same time to use PBR effectively.I don't know if either of these threads came to/near fruition. With Lucas, Rons, or Elfalcons work.
IBL:
www.garagegames.com/community/forums/viewthread/130483
SSDO:
www.garagegames.com/community/forums/viewthread/136081
#77
I think the issue was the framerate with one, the other I'm not totally for sure. Lukas I believe may have been a little further and would be a good place to start with that.
That's where I believe having D3D 11/Open Gl 3+ would help allow for extra power with those type of shaders, at least on the real time gameplay side!
@ Raa, could you put a link to the visual shader system..You were referring too?
06/09/2014 (4:27 pm)
Yes not for sure I believe Ron and Lukas both ran into a similar issue with both of the codes.I think the issue was the framerate with one, the other I'm not totally for sure. Lukas I believe may have been a little further and would be a good place to start with that.
That's where I believe having D3D 11/Open Gl 3+ would help allow for extra power with those type of shaders, at least on the real time gameplay side!
@ Raa, could you put a link to the visual shader system..You were referring too?
#78
I partly agree.
Here why, yes the new versions of DirectX and OpenGL will provide extra power for rendering, but when consider that PBR as been in use since the early 2000's and been used with DirectX 8.2 and OpenGL 1.3.
Yes back then they were using the most power CPU's around and running SCSI RAID in development of CGI. Matter of Fact I happen to have an identical PC that Peter Jackson's CGI team used when they make Lord of The Rings. Dual Pentium Pro 200Mhz, 4Gb RIMM memory and RAID5 SCSI, and a 4x AGP card.
So the version of DirectX or OpenGL has no major affect to PBR other than how fast and maximum resolution it can render the graphic too.
06/09/2014 (5:05 pm)
@KoryI partly agree.
Here why, yes the new versions of DirectX and OpenGL will provide extra power for rendering, but when consider that PBR as been in use since the early 2000's and been used with DirectX 8.2 and OpenGL 1.3.
Yes back then they were using the most power CPU's around and running SCSI RAID in development of CGI. Matter of Fact I happen to have an identical PC that Peter Jackson's CGI team used when they make Lord of The Rings. Dual Pentium Pro 200Mhz, 4Gb RIMM memory and RAID5 SCSI, and a 4x AGP card.
So the version of DirectX or OpenGL has no major affect to PBR other than how fast and maximum resolution it can render the graphic too.
#79
06/09/2014 (5:40 pm)
WETA used a huge renderwall made up of hundreds of computers and by the end of the trigology it had a few thousand computers and was listed in the top 500 supercomputers.
#80
06/09/2014 (6:13 pm)
Pierre..That's what I meant..lol!! Anyways the ground work of planning sound good, just from the brief knowledge I have on the subject. Looks like you guys are looking in the right places (engine wise). Hopefully the D3D 11 and Open Gl ports will come along and finish sometime soon, so we could get some realtime games using PBR powered by T3D :) 
Azaezel
singleton Material(mat_base_metal1) { specularPower[0] = "64"; }github.com/GarageGames/Torque3D/blob/development/Engine/source/materials/materia... takes the variable embedded in the script block, and stores it in the class.
github.com/GarageGames/Torque3D/blob/development/Engine/source/materials/process... tells it when rendering a given material to insert that data into a shader constant register to be passed along.
github.com/GarageGames/Torque3D/blob/development/Engine/source/materials/process... +
github.com/GarageGames/Torque3D/blob/development/Engine/source/shaderGen/shaderG... associates a constant register with a handle.
github.com/GarageGames/Torque3D/blob/development/Engine/source/lighting/advanced... + github.com/GarageGames/Torque3D/blob/development/Engine/source/lighting/advanced... tells the shadergen when cooking up a new shader to (taking this from a resulting hlsl file):
uniform float specularStrength; void main() {... float specularPower = specularPower; float specularStrength = specularStrength; //------------------------- ... // Pixel Specular [Deferred] float specular = pow( abs(d_specular), max((specularPower / AL_ConstantSpecularPower),1.0f)) * specularStrength; col += float4( specularColor.rgb, 0 ) * specular;(note the correspondence of that last chunk with github.com/GarageGames/Torque3D/blob/development/Engine/source/lighting/advanced... when it comes to how to plug in various handles into an inserted line)That clear up enough of how to go from point A to point B, or is more clarification requested?