Solid Line Around Objects Silhouette

#1 do a search for "cel shader", I think I remember seeing something like what you described

#2 Well... you should use PostEffect to do the silhouette rendering, but you need an input texture that defines the shape that needs outlining.

The best bet to get that would be adding a RenderTexTargetBinManager which will render only your selected objects to a R32F texture.

Right now you would have to make that work via C++ changes, but i think we could work out how to make some improvements to RenderTexTargetBinManager and possibly CustomMaterial to allow something like this to work all from script.

Anyway... i don't have time to walk you thru this... but maybe this at least gets you started.

#3 Thanks Tom, I know your busy and I appreciate your input. It will help get me started.

#4 I was able to get a good outline shader by using code from the anti-aliasing shader.


It's a postFX screen-wide shader though. Not sure how you would constrain the outline to a single model.
Still, here's the code for it just in case:

edgeDetect.cs (goes in core/scripts/client/postFx)

singleton GFXStateBlockData( PFX_DefaultOutlineStateBlock )
{
   zDefined = true;
   zEnable = false;
   zWriteEnable = false;
      
   samplersDefined = true;
   samplerStates[0] = SamplerClampLinear;
   //samplerStates[1] = SamplerWrapPoint;
};

singleton ShaderData( PFX_OutlineShader )
{   
   DXVertexShaderFile 	= "shaders/common/postFx/postFxV.hlsl";
   DXPixelShaderFile 	= "shaders/common/postFx/outlineShaderP.hlsl";
         
   //OGLVertexShaderFile  = "shaders/common/postFx/gl//postFxV.glsl";
   //OGLPixelShaderFile   = "shaders/common/postFx/gl/passthruP.glsl";
      
   samplerNames[0] = "$inputTex";
   
   pixVersion = 3.0;
};

singleton ShaderData( PFX_OutlineEdgeDetectShader )
{   
   DXVertexShaderFile 	= "shaders/common/postFx/postFxV.hlsl";
   DXPixelShaderFile 	= "shaders/common/postFx/outlineDetectP.hlsl";
         
   //OGLVertexShaderFile  = "shaders/common/postFx/gl//postFxV.glsl";
   //OGLPixelShaderFile   = "shaders/common/postFx/gl/passthruP.glsl";
      
   samplerNames[0] = "$inputTex";
   
   pixVersion = 3.0;
};

singleton PostEffect( OutlineFX)
{
   renderTime = "PFXAfterDiffuse";
   //renderBin = "ObjTranslucentBin";      
   //renderPriority = 0.1;
      
   shader = PFX_OutlineEdgeDetectShader;
   stateBlock = PFX_DefaultOutlineStateBlock;
   texture[0] = "#prepass";
   target = "$outTex"; // "$backBuffer";
   
   new PostEffect()
   {
      shader = PFX_OutlineShader;
      stateBlock = PFX_DefaultOutlineStateBlock;
      texture[0] = "$inTex"; 
      texture[1] = "$backBuffer";
      target = "$backBuffer";
   };
};

outlineDetectP.hlsl (goes in shaders/common/postFX)

#include "postFx.hlsl"
#include "shadergen:/autogenConditioners.h"

// GPU Gems 3, pg 443-444
float GetEdgeWeight(float2 uv0, in sampler2D prepassBuffer, in float2 targetSize)
{
   float2 offsets[9] = {
      float2( 0.0,  0.0),
      float2(-1.0, -1.0),
      float2( 0.0, -1.0),
      float2( 1.0, -1.0),
      float2( 1.0,  0.0),
      float2( 1.0,  1.0),
      float2( 0.0,  1.0),
      float2(-1.0,  1.0),
      float2(-1.0,  0.0),
   };
   
   
   float2 PixelSize = 1.0 / targetSize;
   
   float Depth[9];
   float3 Normal[9];
   
   for(int i = 0; i < 9; i++)
   {
      float2 uv = uv0 + offsets[i] * PixelSize;
      float4 gbSample = prepassUncondition( tex2D(prepassBuffer, uv) );
      Depth[i] = gbSample.a;
      Normal[i] = gbSample.rgb;
   }
   
   float4 Deltas1 = float4(Depth[1], Depth[2], Depth[3], Depth[4]);
   float4 Deltas2 = float4(Depth[5], Depth[6], Depth[7], Depth[8]);
   
   Deltas1 = abs(Deltas1 - Depth[0]);
   Deltas2 = abs(Depth[0] - Deltas2);
   
   float4 maxDeltas = max(Deltas1, Deltas2);
   float4 minDeltas = max(min(Deltas1, Deltas2), 0.00001);
   
   float4 depthResults = step(minDeltas * 25.0, maxDeltas);
   
   Deltas1.x = dot(Normal[1], Normal[0]);
   Deltas1.y = dot(Normal[2], Normal[0]);
   Deltas1.z = dot(Normal[3], Normal[0]);
   Deltas1.w = dot(Normal[4], Normal[0]);
   
   Deltas2.x = dot(Normal[5], Normal[0]);
   Deltas2.y = dot(Normal[6], Normal[0]);
   Deltas2.z = dot(Normal[7], Normal[0]);
   Deltas2.w = dot(Normal[8], Normal[0]);
   
   Deltas1 = abs(Deltas1 - Deltas2);
   
   float4 normalResults = step(0.4, Deltas1);
   
   normalResults = max(normalResults, depthResults);
   
   return dot(normalResults, float4(1.0, 1.0, 1.0, 1.0)) * 0.25;
}

float4 main( PFXVertToPix IN, 
             uniform sampler2D prepassBuffer :register(S0),
             uniform float2 targetSize : register(C0) ) : COLOR0
{
   return GetEdgeWeight(IN.uv0, prepassBuffer, targetSize );
}

outlineShaderP.hlsl (goes in shaders/common/postFX)

#include "postFx.hlsl"
#include "shadergen:/autogenConditioners.h"

float4 main( PFXVertToPix IN, 
             uniform sampler2D edgeBuffer :register(S0), uniform sampler2D backBuffer : register(S1) ) : COLOR0
{
   float4 e = float4( tex2D( edgeBuffer, IN.uv0 ).rrr, 1.0 );
   e.r = e.g = e.b = 1.0 - e.r;
   return tex2D(backBuffer, IN.uv0) * e;
}

Then just open up the console in-game and type

OutlineFX.enable();

Not exactly what you need, but I hope that helps some.

#5 @MikeJ: Nice work!

#6 Great job with that PostEffect Mike!

We should start a library of PostEffects somewhere. :)

#7 Very Cartoonish look with that shader
Really nice

#8 I think Tom's idea for a PostEffects library is awesome!
seconded.

#9 PostEffects Library third/thirded/double-seconded whichever one is grammatically correct anyhow...
Nice edge shader!

#10 That's a cool effect - is there anyway to detect that the outline is under the water, and not display it unless the camera was under water? Or is it possible to do a post effect before the water is drawn? (goofy, I know)

#11 @Jaimi: yes, both are possible. The PostFX shaders have access to the water plane information, and by changing the renderBin of the postFX you can have it render before the water and other transparent effects.

#12 OMG AWSOME! Thank you so much! If anyone has any ideas about how to isolate this to a specific object please let me know.

#13 I'd render the object(s) you want to have edge highlighting into a separate buffer, and run the edge detction on them. As it stands now, all objects drawn to the g-buffer will get edge processed. You could try masking out the edge overlay, but I think that there will be glitches.

#14 Hmm, the idea of a PostFX library sounds cool.

Nice effect by the way!

#15 +1 for the PostFX library!

#16 @Pat - What if you could add a RenderTexTargetBinManager from script, give it a format and target name, then be able to assign which objects get rendered to it.

Sounds like it would let you compose special buffers which can be used as inputs to PostEffects, but i don't have a clue about how to expose it to script well.

#17 So i'm a little confused about using Post Effects to get the outline on the models silhouette.

From what I understand using the renderBinManager will render all of the models on the screen to a single texture.

In addition to doing another render pass on my selected objects I will also need to do another pass on that texture to JUST get the outline (since post effects are on top of the rendered scene).

Also if I have two objects overlapping then a line will be drawn around their merged silhouette, I want to have a unique line for each objects silhouette. So would these mean that I need to do a separate render pass for each object individual rather than using the post effect system?

Or have a completely missed something here?

#18 The outline shader I posted above uses depth/normal information to detect edges. Therefore, even if you rendered two objects overlapping, each would still have their own outline, since the shader would see the distinct difference in each object's normals and depth 'color'.

Read this article, under "Process". Wikipedia explains it better than I. The technique is the same, just applied to a few models rather than the entire scene.

So basically, you just need to render every model you want an outline around to a single texture, then apply the outline shader to said texture.

#19 @Greg - As of beta2 and possible for the first Torque 3D release... what your wanting will take some custom C++ code.

A few quick thoughts on how i would do it...

1. Add a SimObjectId to MeshRenderInst and set it from TSMesh when it creates and submits the render instance. This allows us to identify the mesh being rendered.

2. Add a new render bin based on RenderTexTargetBinManager. Maybe copy RenderPrePassMgr and start hacking it to get a bit that renders only selected SimObjectIds from RIT_Mesh.

3. Add some script methods to your special renderbin which allows you to add/remove/clear SimObjects you want it to render to the bin.

4. Copy the PostEffect Mike posted above... have it read from the texture generated from your custom renderbin which only has your selected objects.

This would give you a system where you can mark any SceneObject that is based on a DTS as selected and it would render outlined.

Quote:Also if I have two objects overlapping then a line will be drawn around their merged silhouette

Not exactly sure i follow you.

If you look at Mike's image above the player is overlapping the building behind it yet they both have their own silhouette.

#20 Thanks so much for you help Mike & Tom.

My ignorance of all of this is obviously showing, but this is has been a big help.

I should be able to follow your steps and get the effect only on the selected objects Tom.

One thing that Mike's shader does though, that I don't want mine to do, is render lines inside the objects silhouette. So if there is a hard edge on a mechanical model or building it will draw a line on it much like the lines being drawn inside this models silhouette:


I ONLY want the line around the OUTSIDE of the silhouette not inside. I think I should be able to adjust some of the parameters inside the shader to achieve this, or perhaps I need to adjust the depth pass render, if you have any tips on the best way to accomplish this that would be great.


I actually worked on another project in TGE and what we did to get the effect was to simply render out a solid color version of the model with some edge padding and remove any pixels that weren't fully opaque and then rendered the normal model over the top of it. This resulted in a solid line around the outside of the model.

Unfortunately that was on in openGL and the programmer who worked on it no longer works with us.

Also, I think this is why I was getting confused about the silhouettes getting merged because I was visualizing the render passes as silhouettes only (a completely black model) and not traditional a depth pass.

Anyways, thanks again guys your responses have been extremely helpful. Oh, and just for clarification I am an artist by training. I'm working on this with a programmer at work so sorry if I sound so ignorant.