Post-processing madness

So I wanted to touch on another project that I've been working on, and is likely to be the first "pack" I deliver up to my fellow Indie monsters.

There are several aspects to this, but for a quick explanation, it's a scriptable manager for post-processing effects in Torque. Effects can be defined in TorqueScript, chained together, and enabled or disabled in script.

Here's a quick video of some of the effects in action (first time trying to embed a video here, hopefully it works):






---

And here is a couple of examples of how they would be defined in script. First the simplest example, which is the Black and White effect:

////////////////////////
//Black and White
////////////////////////
new ShaderData( BlackAndWhiteShader: BasePostShader )
{
    DXPixelShaderFile 	= "shaders/aegis/post/BlackAndWhite_P.hlsl";
};

new CustomMaterial(BlackAndWhiteMaterial : BasePostMaterial )
{
    shader = BlackAndWhiteShader;
};

new PostLayerPass( BWPass )
{
    finalOutput = true;
    material = BlackAndWhiteMaterial;
};

new PostLayer(BWLayer)
{
    passes[0] = BWPass;
    disabled = true;
};

This should be pretty simple to understand. First you would define the shader/material that will be used for the effect using the standard CustomMaterial setup. Then add a pass which specifies the material that you'll be using, then create a layer with that as the single pass. You'll usually want to have them disabled by default.

No big deal. Now here's a more complex example, which defines the "bloom" effect. I'll leave out the material definitions to keep it a little shorter.

function BloomBlendMaterial::initShaderParams(%this)
{
    %this.setFloatConst( "OriginalImageWeight", 1.0 );
    %this.setFloatConst( "BlurWeight", 0.65 );
}


new PostTextureDesc( Bloom_RT0 )
{
    useTargetWidth = true;
    useTargetHeight = true;
};

new PostTextureDesc( Bloom_RT1 )
{
    useTargetWidth = true;
    useTargetHeight = true;
    targetWidthScale = 0.25;
    targetHeightScale = 0.25;
};

new PostTextureDesc( Bloom_RT2 )
{
    useTargetWidth = true;
    useTargetHeight = true;
    targetWidthScale = 0.25;
    targetHeightScale = 0.25;
};


////////////////
// render passes
////////////////
new PostLayerPass( BloomInput )
{
    targetInput[0] = input;
    targetOutput = 0;
};

new PostLayerPass( BloomBright )
{
    targetInput[0] = 0;
    targetOutput = 1;
    material = BrightMaterial;
};

new PostLayerPass( BloomBlurV )
{
    targetInput[0] = 1;
    targetOutput = 2;
    material = BlurVMaterial;
};

new PostLayerPass( BloomBlurH )
{
    targetInput[0] = 2; 
    targetOutput = 1;
    material = BlurHMaterial;
};

new PostLayerPass( BloomBlend )
{
    targetInput[0] = 0;
    targetInput[1] = 1;

    finalOutput = true;

    material = BloomBlendMaterial;
};

new PostLayer(BloomLayer)
{
    textures[0] = Bloom_RT0;
    textures[1] = Bloom_RT1;
    textures[2] = Bloom_RT2;

    passes[0] = BloomInput;
    passes[1] = BloomBright;
    passes[2] = BloomBlurV;
    passes[3] = BloomBlurH;
    passes[4] = BloomBlend;

    disabled = true;
};

Now I'll go down the list and describe what each part does.

First is BloomBlendMaterial::initShaderParams. This is part of an addition to the material system that makes these things easier. When a material instance is created, this callback is called and you have a chance to setup shader parameters. There's also an updateShaderParam which is called per frame for setting up dynamic shader parameters, I'll show an example of that in a minute.

Next we have a PostTextureDesc. PostTextureDesc are basically render targets that are used by the framework. The first one is a basic render target that uses the current viewport dimensions. The next 2 are scaled to 1/4 the size of the viewport. You can also set a custom size with width and height settings. You can also define the pixel format of the render target, by default it uses your basic R8G8B8X8 format. Here we use one full-size render target, and 2 1/4 size render targets for fast downsampling.

PostLayerPass defines a render pass. The first one, BloomInput, takes the input into the scene and assigns it to the first render target. The input in this case is either the rendered scene, or the output from the previous effect in the chain.

The second pass, BloomBright, takes the input render target as input, and renders a full-screen quad using the provided material. RT0 is passed into the shader as the first sampler. The result is rendered to RT 1.

The third pass takes the output from BloomBright and renders it to RT 2, the fourth pass takes the output from that as it's input and renders it to RT 1.

The final pass takes our original input target as the first sampler, and RT 1 as the second sampler, and renders the final quad to the output target.

The result of this particular effect is that it takes the original scene, downsamples it to 1/4 the size, applies a brightness and a blur, and then blends that back on top of the original scene, for a light bloom effect.

As you can see, this gives you a lot of flexibility on the types of effects that you can render with it.


As I mentioned previously, there's also an "updateShaderParams" method for the custom materials, which could look like this:

function OldMovieMaterial::updateShaderParams(%this)
{
    %this.setTimeConstX( "timeData", 20 );
}

Here, each frame, when the OldMovieMaterial is setting up a pass this callback is called and the "timeData" constant is set to the elapsed time since the last frame in milliseconds, multiplied by 20. I added a lot of these custom constant methods to avoid having to do many calculations in script. For instance there are methods to apply an fmod, sin, cos, etc on the time elapsed. Also special methods for setting gaussian distribution weights and offsets for the gaussian blur effect. Extending these should be pretty simple.

Since many effects require the use of volume textures, I also added support for loading DDS volume textures. And I added a few built-in procedural volume textures that can be used by the effects.

Many of these things will be useful for more than just the post processing effects.

-----------------------

This is going to be the "basic" offering, which will be pretty simple to integrate into your existing code, and fairly powerful in it's own right.

There's also going to be advanced offering, which is much more powerful but will require more work integrating, as it reconfigures the material system and the rendering system.

I'll go into more detail on that, plus add a video of some of the things possible with it, in my next blog. But in a nutshell it's going to be a complete scripted multi-pass rendering framework. It takes the post processing framework, and adds the ability to do more than just process render textures, but also adds the ability to do additional scene renders or partial scene renders. This includes a material scheme system, for fine control of how each material will be rendered during each rendering pass.

Anyway, more info on that later.

Cheers