Custom Shader on T3D?

#21 Yes, u are right. But i'm still far from that developed on RenderMonkey.
After doing some other small improvements, i also recorded a video.

NOTE: If anyone knows how to import the vectors and matrixes regarding the lights (like LightVec, LightColor, LightPosition), please intervening.

#22 Your light vector and your tangent vectors need to be in the same space. If these vectors are in object space and your lighting vector is in world space then you either need to transform your vectors into world space or transform your lighting vector into object space. I would transform the lighting vector since it is less intensive than translating 3 vectors.
Also pay attention to the order of your parameters,since it does matter if it is a row or a column vector.

float3x3 TBN = {IN.T,IN.B,IN.N};
float4 worldVertexPos = mul( objTrans,IN.pos);
float3 LightVec = LightPos.xyz - worldVertexPos.xyz;
L = mul(L,TBN);
L = normalize(L);

Also check out these tutorials:
http://www.liman3d.com/tutorials.html

#23 Generally tools like FX Composer generate shaders which are either incompatible or impossible in a game engine. In particular when you try to make a forward rendered shader work in a deferred rendering engine like Torque you have lots of problems.

As Alfio has found... its not a cut and paste job... it can often become a complete rewrite of the shader. In particular a skin shader, which is basically a different lighting algorithm, doesn't translate across to a deferred renderer which wants to unify lighting into a screen space calculation.

So your options are:

1. Not support deferred rendering on skin surfaces... only do forward rendering.
2. Develop a skin shader that can work with the light prepass buffer and deferred techniques.

In Alfio's case #1 might be his best bet. Make sure you set the 'forwardLit' property on the CustomMaterial and use the forward lighting shader constants (read shaderscommonlighting.hlsl).

The big problem... you'll get no shadows. Which when you have your character standing in a dark corner of the room with glowing skin... will be a problem.

To do #2 you need to really implement a deferred algorithm which requires actual knowledge of the light prepass technique and the math involved in skin shading.

T3D already has a deferred sub surface scattering feature meant for skin... its not that great, but i'd like to see us improve it.

The latest technique for doing skin shading does it as a post effect using stencil to mark the skin pixels on the screen. Like in Wolfgang's Rawk Demo.

#24 How Tom said, is truly complex. I think, i can create a template for RenderMonkey, but before i have yet to solve some problems. Currently, i solved part of the problem of the lights, and vectors. Now my shader receives the right light, at least until there is only one light in the scene. So i have to find a solution to manage multiple lights at once. Another problem, which is about normal, tangent, and binormal.



A solution quite concrete, concerns the use of lightinfo. But this requires a completely different approach, and in any case i do not see how to correlate with the authoring software like RenderMonkey.

or

Something similar to numLights, and a loop on every light, could solve the problem. But in any case would require indexing the lights and get into the hands really big array in complex scenes.

#25 Here are the latest results. Finally, the vectors seem to be nearly correct, although i have some problem with the TBN. Doing working in harmony DefferedRendering and calculated values from my functions, i was able to receive more light and to cast all the shadows correctly. Surely after i realize what it still does not work, but let's say that i have made some progress. :-P

NOTE: In this video i also used a PostFX shader for color and saturation correction.

#26 So, you are going with #2 of Toms suggestion?

#27 Part of my VS:

#define IN_HLSL  
#include "shadergen:/autogenConditioners.h"
#include "hlslStructs.h"   
#include "lighting.hlsl"

// Structures                                                                  
struct ConnectData
{
   float4 Position      : POSITION;
   float2 TexCoord      : TEXCOORD0;
   float3 EyeVec        : TEXCOORD1;
   float3 LightVec      : TEXCOORD2;
   float4 LightCol      : TEXCOORD3;
   float4 screenspacePos: TEXCOORD4;
   float3 WorldNormal   : NORMAL;
   float3 WorldTangent  : TANGENT;
   float3 WorldBinormal : BINORMAL;
};

// Main                                                                        
ConnectData main( VertexIn_PNTTTB IN,
	uniform float4x4 modelview,
        uniform float3 eyePos,
	uniform float4x4 objTrans,
	uniform float4 inLightPos[3],
	uniform float4 inLightColor[4]
)
{
   ConnectData OUT;

   float3 WorldSpacePos = mul(objTrans, IN.pos).xyz;

   // Calculate the Normal, Tangent, and Binormal
   /*
   OUT.WorldNormal = normalize( IN.normal );

   float3 c1 = normalize(cross( OUT.WorldNormal, float3(0.0, 0.0, 1.0) )); 
   float3 c2 = cross( OUT.WorldNormal, float3(0.0, 1.0, 0.0) ); 
   if( length(c1) > length(c2) ) {
      OUT.WorldTangent = c1;
   } else {
      OUT.WorldTangent = c2;
   }
   OUT.WorldBinormal = normalize( cross(OUT.WorldNormal, OUT.WorldTangent) );
   */

   OUT.WorldTangent = mul(objTrans, IN.T).xyz;
   OUT.WorldBinormal = mul(objTrans, cross(IN.T, IN.normal)).xyz;
   OUT.WorldNormal = mul(objTrans, IN.normal).xyz;

   // Calculate the EyeVec
   OUT.EyeVec = -(IN.pos.xyz - eyePos);  

   // Calculate the LightVector
   float3 LightPosition = float3(inLightPos[0].x, inLightPos[1].x, inLightPos[2].x);
   OUT.LightVec = LightPosition - WorldSpacePos;

   // Calculate the LightColor
   OUT.LightCol = 1; //float4(inLightColor[0].x, inLightColor[0].y, inLightColor[0].z, inLightColor[0].a);

   // Texture Coordinate
   OUT.TexCoord.xy = IN.uv0;

   // Vertex Position
   OUT.Position = mul(modelview, float4(IN.pos.xyz, 1));

   // Deferred RT Lighting  
   OUT.screenspacePos = OUT.Position;

   return OUT;
}

Part of my PS:

#include "shadergen:/autogenConditioners.h"
#include "torque.hlsl"

uniform sampler DiffSampler: register( S0 );
uniform sampler NormalSampler: register( S1 );
uniform sampler SpecSampler: register( S2 );
uniform sampler lightInfoBuffer : register(S3);

uniform float4 rtParams3 : register(C0);

....

struct ConnectData
{
   float4 Position      : POSITION;
   float2 TexCoord      : TEXCOORD0;
   float3 EyeVec        : TEXCOORD1;
   float3 LightVec      : TEXCOORD2;
   float4 LightColor    : TEXCOORD3;
   float4 screenspacePos: TEXCOORD4;
   float3 WorldNormal   : NORMAL;
   float3 WorldTangent  : TANGENT;
   float3 WorldBinormal : BINORMAL;
};

....

float4 main( ConnectData IN ) : COLOR
{
   float2 TexUV = IN.TexCoord.xy;

   float3 normal = normalize(tex2D(NormalSampler, TexUV).xyz) * 2.0 - 1.0 ;
   
   float3 WN = IN.WorldNormal;
   float3 TN = IN.WorldTangent;
   float3 BN = IN.WorldBinormal;

   float3 N = (WN * normal.z * 2) + (normal.x * BN + normal.y * -TN);
   N = normalize(N);
     
   float3 EV = normalize(IN.EyeVec);
   float3 LV = normalize(IN.LightVec.xyz);

   float4 DotLN = dot(LV, N);
 
   ....

   float4 BaseLighting = float4(tex2D(NormalSampler, TexUV), 1);

   //---------------------------------------------------------------------------------
   // Deferred RT Lighting
   //---------------------------------------------------------------------------------

   float2 uvScene = IN.screenspacePos.xy / IN.screenspacePos.w;  
   uvScene = ( uvScene + 1.0 ) / 2.0;  
   uvScene.y = 1.0 - uvScene.y;  
   uvScene = ( uvScene * rtParams5.zw ) + rtParams5.xy;  
   float3 d_lightcolor;  
   float d_NL_Att;  
   float d_specular;
   lightinfoUncondition(tex2D(lightInfoBuffer, uvScene), d_lightcolor, d_NL_Att, d_specular);  

   //---------------------------------------------------------------------------------
   // <- Deferred RT Lighting
   //---------------------------------------------------------------------------------
   
   Spec = float4(tex2D(SpecSampler, TexUV), 1) * d_specular;

   float4 LightingOutput =  BaseLighting * (float4(d_lightcolor, 1)) + Spec * d_NL_Att;
   return hdrEncode(LightingOutput);
}

#28 Solved also the problem of Normal, Tangent, and Binormal on VS:

....

   ConnectData OUT;  
  
   float3 WorldSpacePos = mul(objTrans, IN.pos).xyz;  
  
   // Calculate the Normal, Tangent, and Binormal  

   OUT.WorldNormal = normalize( IN.normal );

   float3 c1 = cross( OUT.WorldNormal, float3(0.0, 0.0, 1.0) ); 
   float3 c2 = cross( OUT.WorldNormal, float3(0.0, 1.0, 0.0) ); 

   if( length(c1) > length(c2) )
   {
      OUT.WorldTangent = c1;
   } else {
      OUT.WorldTangent = c2;
   }

   OUT.WorldBinormal = cross(IN.T, IN.normal).xyz;
  
   // Calculate the EyeVec  
   OUT.EyeVec = -(IN.pos.xyz - eyePos);   

   ....

Without MicroDetail:


With MicroDetail:

#29 This shader is looking awesome, I've been following it since the start of the thread and just wanted to say you doing a fantastic job!

#30 Great work man - keep it up!

What'd be really nifty is if RenderMonkey had a toggle for forward/deferred rendering with lighting. Then you could generate and test either one before export. I know, pipedream....

#31 Continuing the study of the CustomShader, i found another problem, at the moment unsurmountable for me. In practice, when i go to set the alpha channel into the PixelShader, i get this result:


As you can see, the objects behind the CustomMaterial do not appear. And in their place you see the clouds.

Uncle Tom, can you explain why? XD

My StateBlock:

singleton GFXStateBlockData( SimpleStateBlock )
{
   blendDefined = true;
   blendEnable = true; 
   blendSrc = GFXBlendSrcAlpha;
   blendDest = GFXBlendInvSrcAlpha;
   
   cullDefined = true;
   cullMode = "GFXCullCCW";
};

#32 You have to mark your material as translucent.

#33 @Ivan: Thanks Ivan. I thought that was not requested in a customshader.

#34 Yes,translucent objects are rendered through the RenderTranslucentMgr bin,therefore you have to mark your material as translucent in order to add your render instance.

#35 Also note that when your object is marked as translucent that it is forward shaded... it will not get shadows or more than 4 lights.

#36 I think i have solved some of the problems i had concerning the CustomShader. Now i was wondering if i could somehow pass variables directly to my shader. Looking through the code of the various templates, i have noticed that some shaders include the keyword "DEFINES", and that in this way, the shader in question receives the value.

singleton ShaderData( PFX_DOFBlurYShader )
{
   DXVertexShaderFile 	= "shaders/common/postFx/dof/DOF_Gausian_V.hlsl";
   DXPixelShaderFile 	= "shaders/common/postFx/dof/DOF_Gausian_P.hlsl";
   pixVersion = 2.0;      
   defines = "BLUR_DIR=float2(0.0,1.0)";         
};

VertToPix main( PFXVert IN )
{
   VertToPix OUT;
   
   ....

   OUT.uv0 = IN.uv + ( ( BLUR_DIR * 3.5f ) / texSize0 );
   OUT.uv1 = IN.uv + ( ( BLUR_DIR * 2.5f ) / texSize0 );
   OUT.uv2 = IN.uv + ( ( BLUR_DIR * 1.5f ) / texSize0 );
   OUT.uv3 = IN.uv + ( ( BLUR_DIR * 0.5f ) / texSize0 );

   ....

   return OUT;
}

Now. I tried to set the definition of my shader like this:

singleton ShaderData( SkinShader )
{
   DXVertexShaderFile     = "shaders/common/SkinV.hlsl";
   DXPixelShaderFile      = "shaders/common/SkinP.hlsl";   

   ....

   defines = "SpecPower=float(0.2)";

   ....
     
   pixVersion = 3.0;
};

But the shader does not receive the value, then it is not compiled.

#37 Ok, is SemiColons delimited =)

$SkinShader::MicroScale = 50;

singleton ShaderData( SkinShader )
{
   DXVertexShaderFile     = "shaders/common/SkinV.hlsl";
   DXPixelShaderFile      = "shaders/common/SkinP.hlsl";   
  
   defines = "MicroScale=float(" @ $SkinShader::MicroScale @ ");MicroFactor=float(0.1);SpecPower=float(0.2)";
     
   pixVersion = 3.0;
};

But in any case, the change of variable is not managed. So for the moment this method is useful just for set the vectors directly from the script.