ShaderGen bug with alpha
by Tom Spilman · in Torque Game Engine Advanced · 07/13/2007 (4:07 pm) · 1 replies
I ran into an issue where a DTS with alpha in the texture was completely semi-transparent and didn't seem to be obeying the texture alpha. It turned out it was a caused by the Sun ambient color and how the ShaderGen builds features for vertex and ambient colors. Combine this with HDR and floating point surfaces and you would get some really bad results.
First ... the vertex shader was doing this...
The problem here is that 'inLightColor' include the alpha channel from the light... which could be less than 1. Alpha from lights is something i'd argue that no one really wants or expects by default. The fix here was simple... in shaderGen/shaderFeature.cpp:
By changing the order of the statements we keep light alpha from seeping into further operations.
... continued below....
First ... the vertex shader was doing this...
ConnectData main( VertData IN,
uniform float4x4 modelview : register(C0),
uniform float4 inLightColor : register(C25),
uniform float3 inLightVec : register(C24),
uniform float3 eyePos : register(C20),
uniform float3 fogData : register(C22),
uniform float4x4 objTrans : register(C12)
)
{
ConnectData OUT;
OUT.hpos = mul(modelview, IN.position);
OUT.shading = saturate( dot(-inLightVec, IN.normal) );
OUT.shading.w = 1.0;
OUT.shading *= inLightColor;The problem here is that 'inLightColor' include the alpha channel from the light... which could be less than 1. Alpha from lights is something i'd argue that no one really wants or expects by default. The fix here was simple... in shaderGen/shaderFeature.cpp:
void VertLightColor::processVert( Vector<ShaderComponent*> &componentList,
GFXShaderFeatureData &fd )
{
// blah
MultiLine * meta = new MultiLine;
meta->addStatement( new GenOp( " @ = saturate( dot(-@, @) );\r\n", outColor, inLightVec, inNormal ) );
// MOVED DOWN.... meta->addStatement( new GenOp( " @.w = 1.0;\r\n", outColor ) );
meta->addStatement( new GenOp( " @ *= @;\r\n", outColor, inColor ) );
meta->addStatement( new GenOp( " @.w = 1.0;\r\n", outColor ) ); // ... TO HERE
output = meta;
}By changing the order of the statements we keep light alpha from seeping into further operations.
... continued below....
About the author
Tom is a programmer and co-owner of Sickhead Games, LLC.
Associate Tom Spilman
Sickhead Games
Fragout main( ConnectData IN, uniform float4 ambient : register(C3), uniform sampler2D diffuseMap : register(S0), uniform sampler2D fogMap : register(S1) ) { Fragout OUT; OUT.col = IN.shading + ambient; OUT.col *= tex2D(diffuseMap, IN.texCoord);The issue here is that its again adding the ambient color with and alpha channel. Also since its adding it the output alpha value can be as much as 2 before its multiplied with the texture alpha. This was causing a rainbow colored mess when floating point surfaces were used.
Again i argue that the default behavior should be to ignore alpha in shading and lighting calculations. The fix is again in shaderGen/shaderFeature.cpp:
void VertLightColor::processPix( Vector<ShaderComponent*> &componentList, GFXShaderFeatureData &fd ) { // blah // REPLACED .... LangElement *addAmbient = new GenOp( "@ + @", inColor, ambient ); LangElement *addAmbient = new GenOp( "float4( @.rgb + @.rgb, 1 )", inColor, ambient ); // ... WITH THIS output = new GenOp( " @;\r\n", assignColor( addAmbient ) );This change ensures that that only the .rgb components of the shading and ambient are added and alpha is set to one. Then it is only the texture alpha that sets the transparency of the final pixel.
Its arguable that this fix negates the need for the fix in the vertex shader, but i think that having them both output sane values is a better approach to things.