Game Development Community

Physically Based Rendering

by Pierre DragoFire Hay · in Torque 3D Professional · 03/05/2014 (4:39 pm) · 156 replies

I've been slowly reading up on this and wonder if anyone else has looked into this or done any work with it and T3D yet?

For those who don't know what Physically Base Rendering or as some call it Physically Based Shading is here's a topic over at RSI covering what is physically based rendering, this covers most aspects of PBR without getting to technical.

What is PBR or sometimes refered to as PBS or BRDF
The most trivial explanation of a PBR/PBS/BRDF (physical based renderer / physical based shader / bidirectional reflectance distribution function) is that it is the bit of shader code describing how a surface reacts to light. Generally, it is responsible for calculating the specular highlights and diffuse characteristics of the surface material. They are mathematical approximations of how surfaces react to light in the real world. In computer graphics, we try to model the physical world as accurately as possible, but we are constrained by computation. For this reason, the mathematical efficiency of BRDFs is very important. Some of the better known BRDFs - Blinn, Phong and Lambert, for instance - are well known for this reason: they are computationally inexpensive to calculate and intuitive to adjust. However they compromise efficiency for accuracy. If we concern ourselves with more expensive and more accurate models, we uncover a second layer of shading models: Oren-Nayar, Cook-Torrance, Askikhmin-Shirley, etc.
There is no single model that fits every situation, but there are some better than others. The Cook-Torrance model has been shown to be a top performer, when compared against actual acquired BRDF data. Of course, with the good comes the bad and Cook-Torrance is one of the most expensive models to compute. But for overall results, it is hard to beat. So this is our target; a nice implementation of the Cook-Torrance reflectance model. Now this maybe an issue when using Cook-Torrance model on mobiles and consoles due to hardware limitations, so it's a question of which module to use.
What's required?
Base PBR implementation consists of 3 things;
Gamma-correct rendering
  • Shading inputs (textures, light colors, vertex colors, etc.) naturally authored, previewed [li]and (often) stored with nonlinear (gamma) encoding
  • Final frame buffer also uses nonlinear encoding
  • This is done for good reasons
    [li]Perceptually uniform(ish) = efficient use of bits
  1. Legacy reasons (tools, file formats, hardware)
Support for HDR values
[ul]
  • Realistic rendering requires handling values much higher than display white (1.0)
  • Before shading: light intensities, lightmaps, environment maps
  • Shading produces highlights that affect bloom, fog, DoF, motion blur, etc.
  • Cheap solutions exist
  • Good tone mapping (ideally filmic)
    Difference between textures and materials.

    There are few fundamental distinctions between textures and materials. For example, you cannot apply a texture on a static mesh or BSP geometry. Textures have to be a part of a material. The material is what you would use to texture your environment and apply to Static Meshes.

    Here are the differences between a texture and a material:

    Textures is a single file, a 2d static image. It is usually a diffuse, specular or a normal map file that you would create in Photoshop or Gimp, as a tga, tiff, bmp, png file. These can be manipulated photographs, hand-painted textures or textures baked in an application such as xNormals.

    www.worldofleveldesign.com/categories/udk/images/013-texture-vs-material-04.jpg
    Materials are made up of various textures combined together inside a Material Editor(in-engine or 3rd party editor). Materials include various textures and material expressions that creates a network of nodes. The final result is a material you can use to apply on your BSP geometry and on Static Meshes. Materials are what you see rendered in-game.
    It should be noted that Specular[i] map on a texture, is connected to the [i]refractive index and as such describes a physical property. The shading model then varies this reflectance based on view angle and surface roughness. Thus the surface roughness is adjusted to create variety and specular not be varied for a given material.

    www.worldofleveldesign.com/categories/udk/images/013-texture-vs-material-06.jpg
    A video on PBR for Artist

    PBR in Substance by Allegorithmic

    Here's a few examples of PBR in action.

    tri-Ace Technical Demo Trailer 2011 "Physically-based Rendering"

    Star Citizen Avenger PBR
    Useful Links:
    Pixar writing a Cook-Torrance surface Shader
    Pixar writing a BRDF template
    Pixar writing a BRDF template part2
    Pixar CookTorrance Slim Template
    Shading course 2012
    Shading course 2013
    Physically Based Lighting in Black-Ops
    Shader code for physically based lighting
    Basic Theory of Physically-Based Rendering
    Paprika Render
    Mitsuba Render
    PBRT Org.
    Unreal PBR
    Houdini Shaders for physically based rendering (PBR)
    D3DBook:(Lighting) Cook-Torrance
    Specular BRDF Reference by Brian Karis from Epic game


    Books of Interest:
    Physically Based Rendering, From Theory to Implementation
    ShaderX7
    The RenderMan, Shading language Guide

    Edit: Updated with new information and links.
    #81
    06/09/2014 (10:16 pm)
    We need to steer this discussion more towards a realistic implementation. Glamour shots are great but Torque is not a renderer, it's a game engine. We need it to actually be able to run at a reasonable framerate.

    Timmy and I were talking about we're going to try to get it as close to Unreal Engine 4 as possible. Both of us have access to libraries of PBR materials ready for UE4 and he has access to the actual engine and is familiar with the shaders.
    #82
    06/09/2014 (10:32 pm)
    If you are familiar with the Marmoset toolbag, we will be following the metalness map way.
    #83
    06/09/2014 (11:49 pm)
    On the IBL thing, going off of: ict.usc.edu/pubs/Image-Based%20Lighting.pdf and www.pauldebevec.com/Probes/ : seems it's essentially a (not even sure if modified version of the existing, judging by those listed under "Probes in the Vertical Cross Cube Format" ) cubemap.

    Edit: stand corrected: www.polycount.com/forum/showpost.php?p=1832820&postcount=46 2 on a specular-influence lerp. (as a starting point)
    #84
    06/10/2014 (1:50 am)
    Not sure that was directed at me Andrew, but I believe that shot was from a realtime render, most likely Marmoset. I will be the first to admit though, Marmoset is for realtime glamor shots. That said, it still stands that other parts of the engine dealing with lighting will have to change for the PBR to even look correct. UE4 falls back on basically a baked lighting setup "lightmass". Something I don't think you want to implement?


    Thought this quote relevant:
    "The problem is that in a physically based context, without IBL all you really have is very advanced specular highlights. While that's all great, in itself it's not a gamechanger."
    http://forum.unity3d.com/threads/evaluating-unreal-4-at-the-moment-pbr.205762/
    #85
    06/10/2014 (2:16 am)
    @Andrew Mac


    Personally we need to agree on a format and file structure for materials and how to implement them within T3D.

    I already own most of Allegorithmic's material editors and a large material library, as I see it they have one of the best material editors out there and already supported by almost all the big players plus a few of the smaller one too. So I've already put a request through to them about what requires their materials require in engine.

    If they come back with what we need and possibly even SDK to help get things rolling, we might be able to cut the development time down a bit.
    #86
    06/10/2014 (2:31 am)
    Toolbox is the same as Allegorithmic's packages, but as I see it's an editor and renderer, and have not exporter features that Allegorithmic already has in place, only importers and good support for model formats.

    Yes Allegorithic doesn't have an exporter of Torque yet, but has ones for [utl=http://www.allegorithmic.com/technology/integrations]3DMAX Studio, UE4, MAYA, and other[/url]

    The key thing is exporting materials into Torque, and having what PBR/PBS SDK we make or modify working together.

    So sorry Andrew and Timmy, toolbox as I see it, is currently a dead-end.

    As in my last post I have request information from Allegorithic, but I forget to meantion this request also asked about the Substance Engine which;

    Substance Engine allows your games and applications to benefit from the power of Substance, which is said to provide:

    1 Reduce drastically the size of your game: this is for example very useful for the next gen console requirements such as PlayGo.

    2 Use the power of the dynamic texturing at real time, to provide unique visual effects in game.

    3 Substance Engine is integrated by default into the leading tools and software of the industry, providing you an optimal workflow for Unity, UE3/UE4/UDK, 3ds max, Maya and modo.


    But until I get some more information I can't provide much more at the moment.
    #87
    06/10/2014 (2:55 am)
    *Edit post removed. I'll leave you guys to it.
    #88
    06/10/2014 (6:21 am)
    Pierre, inconsequentially your coming off as demanding. I can be this myself, and I understand your enthusiasm. We have to let those that can actually accomplish this like Timmy and Andrew decide what initial route to go to get it up and running with the basics.


    Later on can things like substance, a commercial product (which I own as well), can be integrated. Infact, it can take different hlsl input for its viewer, so in theory if you could find the right compiled hlsl files to use, you could setup substance right now for torque as it currently is. We can have a seperate investigation of this in the art section.

    In short, unless we can directly contribute to this, lets not frustrate those putting their own time to try to make this available to all.

    Timmy, please don't allow our over exuberance to turn you off from this project. I'll sign off from this thread unless any further artist input is required.
    #89
    06/10/2014 (6:32 am)
    Look i don't for one second pretend to be a PBS guru, quite frankly i'm not. I just thought i could help getting the ball rolling, that was all. There are so many other pressing issues with T3D that i'm actively trying to help with, it doesn't bother me in the slightest if nothing gets started on this PBS stuff right away. For example this list here: github.com/GarageGames/Torque3D/issues?milestone=2 this is for T3D 3.6 milestone and if we all want to see T3D progress, stuff on that list needs to be done.
    #90
    06/10/2014 (7:39 am)
    @ Andrew Robinson

    You're picking up what we're picking up as well. This thread has this attitude like "if you're not going to make the best PBR in the industry then don't make it at all" which is discouraging to say the least.

    We're trying to pick goals that are not only realistic but achievable in a reasonable amount of time for people who aren't being paid to do this. A little encouragement instead of complaints can go a long way.
    #91
    06/10/2014 (8:38 am)
    I think a simple implementation (UE4 style) is the best option.

    @Timmy, any help you can give with Github issues will be very grateful :D

    I am sure we will have PBR on Torque3D, it's just a matter of time. The important thing is to be realistic and always have a positive attitude.



    #92
    06/10/2014 (9:27 am)
    Seem to have lost my hangout notes, but trying to stick with the trend of publicly documenting the research: the major difference between metalness and specular map seems to be high-metalness takes the specular color (Currently stored as the rgb of the specular map with the possibility of being modified by that colorpicker alongside it) and does a lerp between the albedo and the light color.

    Likewise the Ambient Occlusion Map (where folks would now put the currently baked in shadows that they just toss directly into the diffuse) from the show and tell testing was pretty much a detail map that's faded in in inverse proportions to what is at present specular highlighting.
    #93
    06/10/2014 (12:54 pm)
    @Andrew R.

    sorry if it appeared that I across as demanding, this isn't the case.

    back to topic. we all know this is a big project as it'll involve a complete rewrite of the material system and shader pipeline to implement the basics.

    I've even spoken with Lorne McIntosh (SSOA and DoF kit developer) and been advised of this, I've asked if he can provide any help on that side of things.

    ideally if we can generate more interest in the community, we may find a few that do have more knowledge of PBR/PBS to help.

    So as the Andrew's have pointed out it maybe time to start looking at a plan of attack. And hopefully when Allegorithmic gets back to me, we maybe able to reduce the list a little with what they can provide.

    as I get time I'll try to update the master post with details and information we have posted here and from other sources also.
    #94
    06/10/2014 (3:23 pm)
    I think PBR is absolutely needed for T3D to remain current... but personally I think the main focus should be on mobile. The mobile market is so much more accessible to indie devs and that's where the money is. I don't think the engine community will really grow until can cross the platform barrier as well as the other options. Trying to make major changes like this to the material system and shader pipeline at the same time as trying to port to OpenGL and Android/iOS platforms is going to make things very tricky.
    Wouldn't a better strategy be to get everyone behind Luis' OpenGL work and then start to develop the new material + shader system as OpenGL-first and try to make it compatible with GLES 3?
    #95
    06/10/2014 (3:41 pm)
    monkeychops
    There are a number of different port going at the same time..The open GL port is coming along so I wouldn't say to not focus on one thing over the other. It would be nice to have development going on a few different thing and once those things complete it gets tested then rolled into a point release..So I agree with the way the SC is doing things. That way with the different branches it gives us the community a chance to test things as well. The more the community test various things and help contribute the quicker certain things can get added to the main branch..I believe I suggested Assimp back in 2012 after messing around with the S2 Engine HD and I seen the plus side of things, even though the current import system wasn't anything I had an issue with. I suggested it and just recently Andrew took the time out to add support and soon it will be added to the main branch..It just show how things can get done, so I feel they are handling things in the right way..What we can do is help contribute information or a helping hand where we can..
    #96
    06/10/2014 (11:35 pm)
    @Monkeychops

    PBR/PBS is separate from OpenGL and DirectX, it is within the core code and crosses a number areas within the core coding, it then tells OpenGL, DirectX and other renders how to render materials.

    So PBR is complete independent of OS or Render used. Thus developing PBR/PBS solely for Mobile OS's isn't required, as has been covered by most in this topic, would help all game platforms, not just PC, Mac or Android, etc...

    Once a PBR/PBS update has been made, then OpenGL would possibly require some fine-tuning to take advantage of what PBR/PBS provides.

    #97
    06/11/2014 (12:25 am)
    While i won't contribute another post to this thread, here are a few before/after pbs screenshots from a game that took a very simple approach to pbs. Is it "uber cool, running at 1fps photo realistic"? Nope, is it better than anything T3D has? Yep. I will say you really need to play it to see the full benefit.

    End result: i.imgur.com/5ZQ42F3.jpg

    Before/after:
    Pic 1: ftr.wot-news.com/wp-content/uploads/2013/12/14.jpg
    Pic 2: ftr.wot-news.com/wp-content/uploads/2013/12/24.jpg
    Pic 3: ftr.wot-news.com/wp-content/uploads/2013/12/82.jpg
    #98
    06/11/2014 (1:06 am)
    @Timmy

    nice images, what engine was that from? I can see it's World of Tanks.

    is it Big World back end and what front end? isn't it Havok engine?
    #99
    06/11/2014 (1:32 am)
    Just been going through the File Formating for PBR input file. some 50 pages worth of document on this alone.

    a little bit from structure of input file

    Almost all directives in a pbrt input file have a direct correspondence with a function in the pbrt API, defined in the files core/api.h and core/api.cpp. The only input file directive that does not directly correspond to a function in the API is the Include statement, which allows other input files to be parsed. Include behaves similarly to the #include directive in C++, except that only the directory that the currently-being-processed
    input file is searched for matching filenames. Of course, a complete pathname or a path relative to the current directory can be specified if appropriate.

    Include "geometry/car.pbrt"


    As I uncover more I'll add what I can.

    But if interested here's the [url=http://www.pbrt.org/fileformat.php]page that has the whole document available. :)
    #100
    06/11/2014 (1:38 am)
    Here's an example of glass materials as given by Matt Pharr and Greg Humphreys. As you can see there are some copyright conditions that have to be followed to show this.

    /*
        pbrt source code Copyright(c) 1998-2012 Matt Pharr and Greg Humphreys.
    
        This file is part of pbrt.
    
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are
        met:
    
        - Redistributions of source code must retain the above copyright
          notice, this list of conditions and the following disclaimer.
    
        - Redistributions in binary form must reproduce the above copyright
          notice, this list of conditions and the following disclaimer in the
          documentation and/or other materials provided with the distribution.
    
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
        IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
        TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
        PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
        HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
        SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
        LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
        DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
        THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
        (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
    
     */
    
    
    // materials/glass.cpp*
    #include "stdafx.h"
    #include "materials/glass.h"
    #include "spectrum.h"
    #include "reflection.h"
    #include "paramset.h"
    #include "texture.h"
    
    // GlassMaterial Method Definitions
    BSDF *GlassMaterial::GetBSDF(const DifferentialGeometry &dgGeom, const DifferentialGeometry &dgShading, MemoryArena &arena) const {
        DifferentialGeometry dgs;
        if (bumpMap)
            Bump(bumpMap, dgGeom, dgShading, &dgs);
        else
            dgs = dgShading;
        float ior = index->Evaluate(dgs);
        BSDF *bsdf = BSDF_ALLOC(arena, BSDF)(dgs, dgGeom.nn, ior);
        Spectrum R = Kr->Evaluate(dgs).Clamp();
        Spectrum T = Kt->Evaluate(dgs).Clamp();
        if (!R.IsBlack())
            bsdf->Add(BSDF_ALLOC(arena, SpecularReflection)(R,
                BSDF_ALLOC(arena, FresnelDielectric)(1., ior)));
        if (!T.IsBlack())
            bsdf->Add(BSDF_ALLOC(arena, SpecularTransmission)(T, 1., ior));
        return bsdf;
    }
    
    
    GlassMaterial *CreateGlassMaterial(const Transform &xform,
            const TextureParams &mp) {
        Reference<Texture<Spectrum> > Kr = mp.GetSpectrumTexture("Kr", Spectrum(1.f));
        Reference<Texture<Spectrum> > Kt = mp.GetSpectrumTexture("Kt", Spectrum(1.f));
        Reference<Texture<float> > index = mp.GetFloatTexture("index", 1.5f);
        Reference<Texture<float> > bumpMap = mp.GetFloatTextureOrNull("bumpmap");
        return new GlassMaterial(Kr, Kt, index, bumpMap);
    }