Shaders, registers and loading textures.
by Lukas Joergensen · in Torque 3D Professional · 01/13/2014 (9:05 am) · 9 replies
Hey guys, as you may be aware of, I'm currently trying to create a hlsl compiler. (Which compiles to .hlsl and not the other way around).
However while I have the basic syntax going, I'm kind of lost regarding registers, and loading in textures. As well I'm not sure how to go about the TorqueScript-side of setting up the shader.
I'm just working with simple PostFx shaders atm.
So my question is this:
How do I know what register a texture is gonna have in the .hlsl code?
I.e. how do I know what {REGISTER_NUM} should be here?
Are there predefined register numbers for some shaders? Or does it start at 0?
I have a simple sample that fills the screen with a single color:
That is all for now, thank you for reading this and possibly helping me out!
However while I have the basic syntax going, I'm kind of lost regarding registers, and loading in textures. As well I'm not sure how to go about the TorqueScript-side of setting up the shader.
I'm just working with simple PostFx shaders atm.
So my question is this:
How do I know what register a texture is gonna have in the .hlsl code?
I.e. how do I know what {REGISTER_NUM} should be here?
uniform sampler2D Var0 : register({REGISTER_NUM});Are there predefined register numbers for some shaders? Or does it start at 0?
I have a simple sample that fills the screen with a single color:
struct ConnData
{
float4 Var0 : POSITION;
float2 Var1 : TEXCOORD0;
float2 Var2 : TEXCOORD1;
float2 Var3 : TEXCOORD2;
float2 Var4 : TEXCOORD3;
float3 Var5 : TEXCOORD4;
};
float4 main(ConnData IN) : COLOR0
{
const float3 Var6 = {0,0,0};
const float3 Var7 = {0,0,0};
float4 Var9 = lerp(float4(Var7, 1),float4(Var6, 1),0.5);
return Var9;
}Can you show me a similar example of filling the screen with a texture?That is all for now, thank you for reading this and possibly helping me out!
About the author
IPS Bundle available at: https://www.winterleafentertainment.com/Products/IPS.aspx
#2
That has some other .hlsl structs..
Anyways the abbreviations tells me something, I believe I have seen them before while working on a GPU-based particle system..
I think it's just different structs for the different inputs the Vertex shaders can get.
Or are you talking about the ": POSITION;" stuff?
01/13/2014 (12:42 pm)
@Ron that file makes no sense to me at all.. Apparently, they meant to gather all .hlsl structs in that file, yet what about: game/shaders/common/postFxThat has some other .hlsl structs..
Anyways the abbreviations tells me something, I believe I have seen them before while working on a GPU-based particle system..
I think it's just different structs for the different inputs the Vertex shaders can get.
Or are you talking about the ": POSITION;" stuff?
#3
http://msdn.microsoft.com/en-us/library/windows/desktop/bb509704(v=vs.85).aspx
s0 = texture unit 0
s1 = texture unit 1
etc
For any other input parameters the shader compiler will automatically determine which register to use. When the shader is loaded in torque it constructs a mapping of the shader constant names via the calls to GFXD3D9Shader::_buildShaderConstantHandles.
>There are a number of 'shortcuts' in that file that make T3D shader constructs, a tad different than standard hlsl structs. If you can figure out what the heck is in there.... and how it all works.... then you are WAY ahead of me
This simply defines some common vertex formats which describes the layout of each vertex (e.g. [POSITION(vec3), TEXCOORD(vec2)]). In code you usually end up defining them via the GFXDeclareVertexFormat and GFXImplementVertexFormat macros.
When you make a GFXPrimitiveBuffer you pass in the vertex format. When locking you'll get a helper object which can iterate over the vertex data using the said vertex format. Finally when you draw, the GFXPrimitiveBuffer will essentially tell the device "draw this vertex data using the said vertex format".
01/13/2014 (2:38 pm)
>How do I know what register a texture is gonna have in the .hlsl code?http://msdn.microsoft.com/en-us/library/windows/desktop/bb509704(v=vs.85).aspx
s0 = texture unit 0
s1 = texture unit 1
etc
For any other input parameters the shader compiler will automatically determine which register to use. When the shader is loaded in torque it constructs a mapping of the shader constant names via the calls to GFXD3D9Shader::_buildShaderConstantHandles.
>There are a number of 'shortcuts' in that file that make T3D shader constructs, a tad different than standard hlsl structs. If you can figure out what the heck is in there.... and how it all works.... then you are WAY ahead of me
This simply defines some common vertex formats which describes the layout of each vertex (e.g. [POSITION(vec3), TEXCOORD(vec2)]). In code you usually end up defining them via the GFXDeclareVertexFormat and GFXImplementVertexFormat macros.
When you make a GFXPrimitiveBuffer you pass in the vertex format. When locking you'll get a helper object which can iterate over the vertex data using the said vertex format. Finally when you draw, the GFXPrimitiveBuffer will essentially tell the device "draw this vertex data using the said vertex format".
#4
AHHH... ok. that makes a bit more sense now. So, in a nutshell the vertex formats that are set up in hlslStructs.h are basically 'shortcuts' for vertex shaders. Assuming we knew what exactly those structs were... as I recall they are not commented at all.
Based on what you are saying, we do not actually need to define in and out structures within each shader file. Instead, we reference or add to hlsl.h when defining these parameters. Hence, #INCLUDE hlsl.h
So, in essence the PostFx are handled separately from standard vert shaders?
Man, every time I start looking deeper into this stuff I get more and more confused.... This is one of those things that is not documented well at all.
Ron
01/13/2014 (4:59 pm)
@James,AHHH... ok. that makes a bit more sense now. So, in a nutshell the vertex formats that are set up in hlslStructs.h are basically 'shortcuts' for vertex shaders. Assuming we knew what exactly those structs were... as I recall they are not commented at all.
Based on what you are saying, we do not actually need to define in and out structures within each shader file. Instead, we reference or add to hlsl.h when defining these parameters. Hence, #INCLUDE hlsl.h
So, in essence the PostFx are handled separately from standard vert shaders?
Man, every time I start looking deeper into this stuff I get more and more confused.... This is one of those things that is not documented well at all.
Ron
#5
I know the basics about how it works, my issue is if I set a sampler in a CustomMaterial it usually looks something like:
How do I know what number these will have? Can I be sure that they wont be rearranged internally, or that the engine itself will add a sampler?
Also a PostFx doesn't use a material, so how would I pass in textures for PostFx's?
@Ron yeah, there is no documentation on this except for the occasional examples and threads. The .hlsl part is fine as it is mostly like other engines, however the connection between the engine and .hlsl is the confusing part.
01/13/2014 (9:32 pm)
@JamesI know the basics about how it works, my issue is if I set a sampler in a CustomMaterial it usually looks something like:
singleton CustomMaterial( SkinShaderMaterial )
{
mapTo = "Skin_defaultMat_Map-COL.jp";
sampler["DiffSampler"] = "art/shapes/SkinShader/Map-1024_COLOR";
sampler["NormalSampler"] = "art/shapes/SkinShader/Map-1024_NRM";
sampler["SpecSampler"] = "art/shapes/SkinShader/Map-1024_SPEC";
sampler["MicroSampler"] = "art/shapes/SkinShader/Default_MicroDetail_NRM";
sampler["TransSampler"] = "art/shapes/SkinShader/Map-1024_OCC";
shader = SkinShader;
//stateBlock = SkinStateBlock;
version = 3.0;
}; (Taken from here)How do I know what number these will have? Can I be sure that they wont be rearranged internally, or that the engine itself will add a sampler?
Also a PostFx doesn't use a material, so how would I pass in textures for PostFx's?
@Ron yeah, there is no documentation on this except for the occasional examples and threads. The .hlsl part is fine as it is mostly like other engines, however the connection between the engine and .hlsl is the confusing part.
#6
Technically speaking you won't absolutely know which order the samplers will be specified since the CustomMaterial iterates through the property field list using SimFieldDictionaryIterator. If you check out that code it iterates through a hash table which depending on how the values are hashed might be in any order.
If you step through the code however you'll find that ShaderConstHandles::init grabs each of these samplers as a constant and stores them in the mTexHandlesSC array. Then in ProcessedCustomMaterial::setTextureStages it sets the texture units based on which sampler was assigned to the constant.
So in short, it doesn't matter what register you assign.
@Ron,
If all else fails, just read through the code and step through in the debugger along with a copy of Direct3D for Dummies. Thats what I did ;)
01/14/2014 (1:05 am)
@Lukas,Technically speaking you won't absolutely know which order the samplers will be specified since the CustomMaterial iterates through the property field list using SimFieldDictionaryIterator. If you check out that code it iterates through a hash table which depending on how the values are hashed might be in any order.
If you step through the code however you'll find that ShaderConstHandles::init grabs each of these samplers as a constant and stores them in the mTexHandlesSC array. Then in ProcessedCustomMaterial::setTextureStages it sets the texture units based on which sampler was assigned to the constant.
So in short, it doesn't matter what register you assign.
@Ron,
If all else fails, just read through the code and step through in the debugger along with a copy of Direct3D for Dummies. Thats what I did ;)
#7
//shader begin
uniform sampler2D Sampler0; // s0 register
uniform sampler2D Sampler1; // s1 register
..etc
Adding features, you define injections - the order changes.
//shader begin
#define DETAIL_MAP
uniform sampler2D Sampler4; // s0 register
#ifdef DETAIL_MAP
uniform sampler2D Sampler7; // s1 register
#endif // DETAIL_MAP
uniform sampler2D Sampler2; // s2 register
..etc
The number 0 is the uniform location.
s0 != 0
The engine commit data using uniform locations.
Basically it doesn't matter what register you assign.
01/14/2014 (4:34 am)
The number s0 is the sampler register number. It is compiler dependent.//shader begin
uniform sampler2D Sampler0; // s0 register
uniform sampler2D Sampler1; // s1 register
..etc
Adding features, you define injections - the order changes.
//shader begin
#define DETAIL_MAP
uniform sampler2D Sampler4; // s0 register
#ifdef DETAIL_MAP
uniform sampler2D Sampler7; // s1 register
#endif // DETAIL_MAP
uniform sampler2D Sampler2; // s2 register
..etc
The number 0 is the uniform location.
s0 != 0
The engine commit data using uniform locations.
Basically it doesn't matter what register you assign.
#8
Try this:
From postEffect.cpp:
01/14/2014 (4:44 am)
PostEffect texture field array are mapped in order to sampler registers in hlsl.Try this:
singleton PostEffect( MyPostFx)
{
shader = MyPostFxShader;
stateBlock = MyPostFxStateBlock;
texture[0] = "#prepass"; //assigned to register(S0)
texture[1] = "$backBuffer"; //assigned to register(S1)
texture[2] = "myText"; //assigned to register(S2)
texture[3] = "myText2"; //assigned to register(S3)
};From postEffect.cpp:
void PostEffect::_setupTexture( U32 stage, GFXTexHandle &inputTex, const RectI *inTexViewport )
{
// texture field definitions.. texture[n]
const String &texFilename = mTexFilename[ stage ];
... // more code here, texFilename used to assign theTex
if ( theTex.isValid() )
GFX->setTexture( stage, theTex ); //linked to register in hlsl
}
#9
I.e.
Thanks all of you! I will see if I can get textures working in my compiler now! I'll make sure to throw a video up on Youtube when I get it working!
Too bad this gives me the need of generating scripts as well, more work, longer production.
I think I've got it all now tho!
Again thanks a lot!
01/14/2014 (6:20 am)
@James, ahh so in the customeMaterial, when you set sampler["DiffSampler"] = "art/shapes/SkinShader/Map-1024_COLOR";It's really the name of the sampler (i.e. "DiffSampler") that matters and which should match the name of the sampler in the shader?
I.e.
uniform sampler2D DiffSampler: register( S0 );
Thanks all of you! I will see if I can get textures working in my compiler now! I'll make sure to throw a video up on Youtube when I get it working!
Too bad this gives me the need of generating scripts as well, more work, longer production.
I think I've got it all now tho!
Again thanks a lot!
Associate Ron Kapaun
3tdstudios.com
If I could reliably figure this part out I would offer some help. The biggest issue would be this file: game/shaders/common/hlslStructs.h
There are a number of 'shortcuts' in that file that make T3D shader constructs, a tad different than standard hlsl structs. If you can figure out what the heck is in there.... and how it all works.... then you are WAY ahead of me.
Ron