Physically Based Rendering
by Pierre DragoFire Hay · in Torque 3D Professional · 03/05/2014 (4:39 pm) · 156 replies
I've been slowly reading up on this and wonder if anyone else has looked into this or done any work with it and T3D yet?
For those who don't know what Physically Base Rendering or as some call it Physically Based Shading is here's a topic over at RSI covering what is physically based rendering, this covers most aspects of PBR without getting to technical.
What is PBR or sometimes refered to as PBS or BRDF
The most trivial explanation of a PBR/PBS/BRDF (physical based renderer / physical based shader / bidirectional reflectance distribution function) is that it is the bit of shader code describing how a surface reacts to light. Generally, it is responsible for calculating the specular highlights and diffuse characteristics of the surface material. They are mathematical approximations of how surfaces react to light in the real world. In computer graphics, we try to model the physical world as accurately as possible, but we are constrained by computation. For this reason, the mathematical efficiency of BRDFs is very important. Some of the better known BRDFs - Blinn, Phong and Lambert, for instance - are well known for this reason: they are computationally inexpensive to calculate and intuitive to adjust. However they compromise efficiency for accuracy. If we concern ourselves with more expensive and more accurate models, we uncover a second layer of shading models: Oren-Nayar, Cook-Torrance, Askikhmin-Shirley, etc.
There is no single model that fits every situation, but there are some better than others. The Cook-Torrance model has been shown to be a top performer, when compared against actual acquired BRDF data. Of course, with the good comes the bad and Cook-Torrance is one of the most expensive models to compute. But for overall results, it is hard to beat. So this is our target; a nice implementation of the Cook-Torrance reflectance model. Now this maybe an issue when using Cook-Torrance model on mobiles and consoles due to hardware limitations, so it's a question of which module to use.
What's required?
Base PBR implementation consists of 3 things;
Gamma-correct rendering
[ul]Realistic rendering requires handling values much higher than display white (1.0) Before shading: light intensities, lightmaps, environment maps Shading produces highlights that affect bloom, fog, DoF, motion blur, etc. Cheap solutions exist Good tone mapping (ideally filmic)
Difference between textures and materials.
There are few fundamental distinctions between textures and materials. For example, you cannot apply a texture on a static mesh or BSP geometry. Textures have to be a part of a material. The material is what you would use to texture your environment and apply to Static Meshes.
Here are the differences between a texture and a material:
Textures is a single file, a 2d static image. It is usually a diffuse, specular or a normal map file that you would create in Photoshop or Gimp, as a tga, tiff, bmp, png file. These can be manipulated photographs, hand-painted textures or textures baked in an application such as xNormals.
Materials are made up of various textures combined together inside a Material Editor(in-engine or 3rd party editor). Materials include various textures and material expressions that creates a network of nodes. The final result is a material you can use to apply on your BSP geometry and on Static Meshes. Materials are what you see rendered in-game.
It should be noted that Specular[i] map on a texture, is connected to the [i]refractive index and as such describes a physical property. The shading model then varies this reflectance based on view angle and surface roughness. Thus the surface roughness is adjusted to create variety and specular not be varied for a given material.
A video on PBR for Artist
PBR in Substance by Allegorithmic
Here's a few examples of PBR in action.
tri-Ace Technical Demo Trailer 2011 "Physically-based Rendering"
Star Citizen Avenger PBR
Useful Links:
Pixar writing a Cook-Torrance surface Shader
Pixar writing a BRDF template
Pixar writing a BRDF template part2
Pixar CookTorrance Slim Template
Shading course 2012
Shading course 2013
Physically Based Lighting in Black-Ops
Shader code for physically based lighting
Basic Theory of Physically-Based Rendering
Paprika Render
Mitsuba Render
PBRT Org.
Unreal PBR
Houdini Shaders for physically based rendering (PBR)
D3DBook:(Lighting) Cook-Torrance
Specular BRDF Reference by Brian Karis from Epic game
Books of Interest:
Physically Based Rendering, From Theory to Implementation
ShaderX7
The RenderMan, Shading language Guide
Edit: Updated with new information and links.
For those who don't know what Physically Base Rendering or as some call it Physically Based Shading is here's a topic over at RSI covering what is physically based rendering, this covers most aspects of PBR without getting to technical.
What is PBR or sometimes refered to as PBS or BRDF
The most trivial explanation of a PBR/PBS/BRDF (physical based renderer / physical based shader / bidirectional reflectance distribution function) is that it is the bit of shader code describing how a surface reacts to light. Generally, it is responsible for calculating the specular highlights and diffuse characteristics of the surface material. They are mathematical approximations of how surfaces react to light in the real world. In computer graphics, we try to model the physical world as accurately as possible, but we are constrained by computation. For this reason, the mathematical efficiency of BRDFs is very important. Some of the better known BRDFs - Blinn, Phong and Lambert, for instance - are well known for this reason: they are computationally inexpensive to calculate and intuitive to adjust. However they compromise efficiency for accuracy. If we concern ourselves with more expensive and more accurate models, we uncover a second layer of shading models: Oren-Nayar, Cook-Torrance, Askikhmin-Shirley, etc.
There is no single model that fits every situation, but there are some better than others. The Cook-Torrance model has been shown to be a top performer, when compared against actual acquired BRDF data. Of course, with the good comes the bad and Cook-Torrance is one of the most expensive models to compute. But for overall results, it is hard to beat. So this is our target; a nice implementation of the Cook-Torrance reflectance model. Now this maybe an issue when using Cook-Torrance model on mobiles and consoles due to hardware limitations, so it's a question of which module to use.
What's required?
Base PBR implementation consists of 3 things;
Gamma-correct rendering
- Shading inputs (textures, light colors, vertex colors, etc.) naturally authored, previewed [li]and (often) stored with nonlinear (gamma) encoding
- Final frame buffer also uses nonlinear encoding
- This is done for good reasons
- [li]Perceptually uniform(ish) = efficient use of bits
- Legacy reasons (tools, file formats, hardware)
[ul]
Difference between textures and materials.
There are few fundamental distinctions between textures and materials. For example, you cannot apply a texture on a static mesh or BSP geometry. Textures have to be a part of a material. The material is what you would use to texture your environment and apply to Static Meshes.
Here are the differences between a texture and a material:
Textures is a single file, a 2d static image. It is usually a diffuse, specular or a normal map file that you would create in Photoshop or Gimp, as a tga, tiff, bmp, png file. These can be manipulated photographs, hand-painted textures or textures baked in an application such as xNormals.
Materials are made up of various textures combined together inside a Material Editor(in-engine or 3rd party editor). Materials include various textures and material expressions that creates a network of nodes. The final result is a material you can use to apply on your BSP geometry and on Static Meshes. Materials are what you see rendered in-game.
It should be noted that Specular[i] map on a texture, is connected to the [i]refractive index and as such describes a physical property. The shading model then varies this reflectance based on view angle and surface roughness. Thus the surface roughness is adjusted to create variety and specular not be varied for a given material.
A video on PBR for Artist
PBR in Substance by Allegorithmic
Here's a few examples of PBR in action.
tri-Ace Technical Demo Trailer 2011 "Physically-based Rendering"
Star Citizen Avenger PBR
Useful Links:
Pixar writing a Cook-Torrance surface Shader
Pixar writing a BRDF template
Pixar writing a BRDF template part2
Pixar CookTorrance Slim Template
Shading course 2012
Shading course 2013
Physically Based Lighting in Black-Ops
Shader code for physically based lighting
Basic Theory of Physically-Based Rendering
Paprika Render
Mitsuba Render
PBRT Org.
Unreal PBR
Houdini Shaders for physically based rendering (PBR)
D3DBook:(Lighting) Cook-Torrance
Specular BRDF Reference by Brian Karis from Epic game
Books of Interest:
Physically Based Rendering, From Theory to Implementation
ShaderX7
The RenderMan, Shading language Guide
Edit: Updated with new information and links.
#2
That's why it's not going to be real-time real soon. You need massive hardware (just go look at SGI's render machinery) to hit anything close, and that's assuming large-scale parallelization of the renderer and heavy-duty memory throughput. The result is nowhere near worth the cost - I'm not spending half a million dollars on my gaming PC....
Reminds me of an exercise from one of my classes on MAX - a shiny copper tea pot on an old-style electric stove with shiny steel drip catchers. I turned on raytraced reflections and set the depth to 4 - and brought my machine to its knees at 90 minutes per frame (30 second animation). I cancelled it the following morning when I realized that the focus of the exercise was the steam, not the kettle. Changed the reflections to environment maps on both metal objects and it was much faster.
03/06/2014 (8:58 am)
Hey Ron - what's the frame render time in MAX for that?That's why it's not going to be real-time real soon. You need massive hardware (just go look at SGI's render machinery) to hit anything close, and that's assuming large-scale parallelization of the renderer and heavy-duty memory throughput. The result is nowhere near worth the cost - I'm not spending half a million dollars on my gaming PC....
Reminds me of an exercise from one of my classes on MAX - a shiny copper tea pot on an old-style electric stove with shiny steel drip catchers. I turned on raytraced reflections and set the depth to 4 - and brought my machine to its knees at 90 minutes per frame (30 second animation). I cancelled it the following morning when I realized that the focus of the exercise was the steam, not the kettle. Changed the reflections to environment maps on both metal objects and it was much faster.
#3
PBR is being used in several game engines.
PBR in Remember Me: "linked removed"
PBR in UE4:"link removed"
A crap ton of pdfs and PowerPoint presentations blog.selfshadow.com/publications/s2012-shading-course/
The MetalGear Solid Fox engine .
Guerrilla Games Killzone: Shadowfall uses it.
Call of Duty: Black Ops advances.realtimerendering.com/s2011/Lazarov-Physically-Based-Lighting-in-Black-...
And yes even Unity is working on this....so really T3d is really behind the power curve not only on this but in several area's.
03/06/2014 (12:31 pm)
@ Ron stop using DX9c That's the your problemPBR is being used in several game engines.
PBR in Remember Me: "linked removed"
PBR in UE4:"link removed"
A crap ton of pdfs and PowerPoint presentations blog.selfshadow.com/publications/s2012-shading-course/
The MetalGear Solid Fox engine .
Guerrilla Games Killzone: Shadowfall uses it.
Call of Duty: Black Ops advances.realtimerendering.com/s2011/Lazarov-Physically-Based-Lighting-in-Black-...
And yes even Unity is working on this....so really T3d is really behind the power curve not only on this but in several area's.
#4
03/06/2014 (8:06 pm)
Sorry - my post was really more in reference to "'real time' raytraced" than to "physically-based" rendering. 
#5
Real-time raytracing is the holy grail of realistic rendering, but it's a long ways off.
What people are talking about when it comes to Physically based Lighting/Rendering is, in short terms, rendering lighting and materials that use energy conservation.
You have a light that outputs X 'energy' and a given material absorbs/reflects a certain amount before getting to the camera, akin to how it works in real life. If the material properties are close to real life, it'll look pretty realistic.
That video is from the GDC keynote covering how the Fox engine works, and it goes through how they do their materials and linear rendering - which is a huge part in how it looks so good.
I've seen some implementations of PBR in various places and at it's core, it's really not very complicated. Getting the reflectance values for various materials is pretty important(iirc, the above GDC keynote links to one database) as well as moving the pipeline into linear space.
If you get both of those, you'll be most of the way there for good looking PBR.
03/06/2014 (10:02 pm)
Yeah, a distinction very definitely needs to be made here.Real-time raytracing is the holy grail of realistic rendering, but it's a long ways off.
What people are talking about when it comes to Physically based Lighting/Rendering is, in short terms, rendering lighting and materials that use energy conservation.
You have a light that outputs X 'energy' and a given material absorbs/reflects a certain amount before getting to the camera, akin to how it works in real life. If the material properties are close to real life, it'll look pretty realistic.
That video is from the GDC keynote covering how the Fox engine works, and it goes through how they do their materials and linear rendering - which is a huge part in how it looks so good.
I've seen some implementations of PBR in various places and at it's core, it's really not very complicated. Getting the reflectance values for various materials is pretty important(iirc, the above GDC keynote links to one database) as well as moving the pipeline into linear space.
If you get both of those, you'll be most of the way there for good looking PBR.
#6
From my research on it it could be done it DX9 (I think) but for the full potential would require moving up to DX 10 or 11...
Then again with DX 12 recently announced maybe that's what should be looked at..
03/07/2014 (11:31 am)
Yes this is one of those things that,IMO would give good variety to T3D than just using the default advanced lighting..From my research on it it could be done it DX9 (I think) but for the full potential would require moving up to DX 10 or 11...
Then again with DX 12 recently announced maybe that's what should be looked at..
#7
@Ron I'm looking into it and possibly use it, I know Houdini 9+ supports it in processing.
03/07/2014 (8:53 pm)
@Jimmy some of those links you posted aren't available.@Ron I'm looking into it and possibly use it, I know Houdini 9+ supports it in processing.
#8
03/07/2014 (9:02 pm)
@Pierre Thanks I removed those links and added another one. 
#9
Unreal do fake PBR, this is not a real real-time raytracer.
In order to have a real raytracer, you need a new class hardware with a new pipeline, having a raytracer module (at least).
03/08/2014 (4:39 am)
PBR is for movies,advertising, not for games:)Unreal do fake PBR, this is not a real real-time raytracer.
In order to have a real raytracer, you need a new class hardware with a new pipeline, having a raytracer module (at least).
#10
Vanduul Scythe Redesigned with PBR:
The new Avenger and PBR Rendering:
03/08/2014 (10:52 am)
@Ivan not true, Star Citizen is using PBR and so are a number of other games. It appears to becoming a more common addition in current games in development at the moment. Yes some are still playing around with it, but as you can see from the two below videos some are doing it well.Vanduul Scythe Redesigned with PBR:
The new Avenger and PBR Rendering:
#11
Instead of current lighting techniques to adjust maps and everything you get the lighting based on light bounces.
This is very heavy to work realtime, that's why engine developers use a predictive set of functions to determine the lighting. Since now I am also a game engine developer at Gameloft, I cannot explain in details how this is implemented, but in short term it is a hard process of baking bounding trajectories.
03/10/2014 (2:43 am)
No, you don't understand what PBR is and how it is implemented.Instead of current lighting techniques to adjust maps and everything you get the lighting based on light bounces.
This is very heavy to work realtime, that's why engine developers use a predictive set of functions to determine the lighting. Since now I am also a game engine developer at Gameloft, I cannot explain in details how this is implemented, but in short term it is a hard process of baking bounding trajectories.
#12
Without giving away to much info. How would it be implemented in a engine like T3D?
Also could you explain a bit of what it is (from what you've learned about it)?
I admit, I may not have as much knowledge on it as you may but we clearly see the method being used real time, so without a doubt I know it could be done. The only issue would be how to go about it..
03/10/2014 (4:22 am)
Ivan,Without giving away to much info. How would it be implemented in a engine like T3D?
Also could you explain a bit of what it is (from what you've learned about it)?
I admit, I may not have as much knowledge on it as you may but we clearly see the method being used real time, so without a doubt I know it could be done. The only issue would be how to go about it..
#13
yes please explain this more, as from what I've seen from RSI in regards to their Star Citizen game all their models are going through a complete redesign for PBR and the Engine they use is being modified to allow for this. I even asked them about this and they said they had to do this now before the Alpha testing got too far ahead to make sure the core engine changes were bug free before going farther ahead in development, to the point they delayed a major Alpha testing release by 4-6 months, as all exiting models had to be completely redone to support PBR coding.
As I understand and please understand I'm still going through all the documentation I can find and books I can purchase, PBR code is hard-coded in to the core of the engine and models must reference materials listed in this code, and depending on that materials properties will govern the way light is reflected off it and/or adsorbed by that part of the model, thus as I understand it you could have a character model made where the visor and metal of it's armour could have a high reflection, why the leather and skin partly adsorbs the light and brightens a little due to this. At the same time any damage marks could have different effects, eg diffusion.
Now this is how I currently understand it and as I said before I'm still reading up on the whole process, lots of maths involved.
But with the power of most PC's these days I can see this become a major bonus to be included in games.
When you could get images like this in game it makes me want to get more knowledge about the whole PBR process.
A good topic to read is Basic Theory of Physically-Based Rendering and Tutorial: Physically Based Rendering, And You Can Too!
(trying to work out the image sizing, so sorry about cut images)
03/14/2014 (3:59 pm)
@Ivan,yes please explain this more, as from what I've seen from RSI in regards to their Star Citizen game all their models are going through a complete redesign for PBR and the Engine they use is being modified to allow for this. I even asked them about this and they said they had to do this now before the Alpha testing got too far ahead to make sure the core engine changes were bug free before going farther ahead in development, to the point they delayed a major Alpha testing release by 4-6 months, as all exiting models had to be completely redone to support PBR coding.
As I understand and please understand I'm still going through all the documentation I can find and books I can purchase, PBR code is hard-coded in to the core of the engine and models must reference materials listed in this code, and depending on that materials properties will govern the way light is reflected off it and/or adsorbed by that part of the model, thus as I understand it you could have a character model made where the visor and metal of it's armour could have a high reflection, why the leather and skin partly adsorbs the light and brightens a little due to this. At the same time any damage marks could have different effects, eg diffusion.
Now this is how I currently understand it and as I said before I'm still reading up on the whole process, lots of maths involved.But with the power of most PC's these days I can see this become a major bonus to be included in games.
When you could get images like this in game it makes me want to get more knowledge about the whole PBR process.
A good topic to read is Basic Theory of Physically-Based Rendering and Tutorial: Physically Based Rendering, And You Can Too!
(trying to work out the image sizing, so sorry about cut images)
#14
03/14/2014 (4:16 pm)
Could somebody please explain Physical Based Rendering to me? This a question asked about PBR to RSI and their answer, might be interesting for some.
#15
These would require Material Attributes with the following number of attributes or parameters to control their appearance;
Material Attributes:
Albedo
Bump
Displacement
Exponent
Base Emission
Emission
Absorption Layer Transmittance
Layer
Material Type:
Diffuse
Phong
Specular
Oren-Nayar
Glossy Transparent
Diffuse Transmitter
Blend
Exit Portal
Coating
Double-Sided Thin
Null
Some of these would require a bit of work, which I've not yet touched on in my investigations.
03/14/2014 (4:44 pm)
As I current understand it PBR will require a database for each material that would be used in a model.These would require Material Attributes with the following number of attributes or parameters to control their appearance;
Material Attributes:
Albedo
Bump
Displacement
Exponent
Base Emission
Emission
Absorption Layer Transmittance
Layer
Material Type:
Diffuse
Phong
Specular
Oren-Nayar
Glossy Transparent
Diffuse Transmitter
Blend
Exit Portal
Coating
Double-Sided Thin
Null
Some of these would require a bit of work, which I've not yet touched on in my investigations.
#16
www.altdevblogaday.com/2011/08/23/shader-code-for-physically-based-lighting/
03/16/2014 (6:31 pm)
Here is another link fwiw.www.altdevblogaday.com/2011/08/23/shader-code-for-physically-based-lighting/
#17
@Kory
This cannot not enter T3D easily, since almost all of the papers are copyrighted and GG have to invest money or at least have to talk with somebody.
03/17/2014 (5:28 am)
Guys, this is under NDA.@Kory
This cannot not enter T3D easily, since almost all of the papers are copyrighted and GG have to invest money or at least have to talk with somebody.
#18
I'm sure it would be some heavy work but having the source for T3D should make it a little easier..
I'm sure it could be done also, if you look at unity. There has been a few attempts and they don't even offer the source (unless you pay the 1500).
Also if the paper are copyrighted, why would that be an issue for GG or T3D?
GG open sourced T3D and no longer doing development, that's were the SC comes in..
I know your under NDA but an idea of what would need to be done may help someone that is trying to implement this system. Nonetheless if I could find a experienced programmer, after my project is out this would be something I'm going to look into having done, if it hasn't been released by then..
03/17/2014 (6:26 am)
Ivan,I'm sure it would be some heavy work but having the source for T3D should make it a little easier..
I'm sure it could be done also, if you look at unity. There has been a few attempts and they don't even offer the source (unless you pay the 1500).
Also if the paper are copyrighted, why would that be an issue for GG or T3D?
GG open sourced T3D and no longer doing development, that's were the SC comes in..
I know your under NDA but an idea of what would need to be done may help someone that is trying to implement this system. Nonetheless if I could find a experienced programmer, after my project is out this would be something I'm going to look into having done, if it hasn't been released by then..
#19
I have to agree with Kory, I currently have sitting on my desk a copy of "Physically Based Rendering, From Theory to Implementation" this has some 1163 pages about the subject and only has legal mention of copyrights about the book being reproduced and nothing about the code being build based on the book being subject to legal rules.
Yes there maybe other sources of information/code out on the web that may have NDA's.
As GG has always mentioned if anyone produces a add-on module it can be added and even sold as an add-on through GG, and with T3D being open source now, well anything can happen without GG's knowledge.
Personally I plan to develop this with the information I have and see where it leads. The reason I started this topic was to see if others had done so yet and if so work together to develop and refine PBR for T3D and in-turn T2D.
03/17/2014 (12:00 pm)
@IvanI have to agree with Kory, I currently have sitting on my desk a copy of "Physically Based Rendering, From Theory to Implementation" this has some 1163 pages about the subject and only has legal mention of copyrights about the book being reproduced and nothing about the code being build based on the book being subject to legal rules.
Yes there maybe other sources of information/code out on the web that may have NDA's.
As GG has always mentioned if anyone produces a add-on module it can be added and even sold as an add-on through GG, and with T3D being open source now, well anything can happen without GG's knowledge.
Personally I plan to develop this with the information I have and see where it leads. The reason I started this topic was to see if others had done so yet and if so work together to develop and refine PBR for T3D and in-turn T2D.
#20
I'm with you, not much of a programmer but I'll try to help out where I can.
With development for DX 11 in works maybe that would help things with adding PBR..
I'll continue to do research and pass on what I find..
03/17/2014 (12:45 pm)
Hey PierreI'm with you, not much of a programmer but I'll try to help out where I can.
With development for DX 11 in works maybe that would help things with adding PBR..
I'll continue to do research and pass on what I find..
Associate Ron Kapaun
3tdstudios.com
GREAT question. I have played around a bit but, based on the current tech (DX9c)... I can say that the results have been less then stellar. I can do some pretty good 'real time' architectural' renders and simple animations.... (using some pretty crazy mapping tech that involves TONS of transparency mapping as well as a bunch of 'shader' cheats.) I can get some stuff that does not look 'bad' but as for 'real time' raytraced physical based renders... no. To be honest, I do not see anything like that in the near future either. This is a game engine after all and not a full on render pipeline. Oh yeah... one last thing... I would love something like real world accurate raytrace rendering... I DEEPLY want my games to look like what I render in 3DS MAX.... but, I am a game dev so, I have to compromise a bit. Heck, I don't think even UnReal or CryTek support anything like that... Honestly, if we dig deep, Unity and even Chrome Engine 5 or 6 can not do that.