Shader Q&a/faq
by Alex Scarborough · in General Discussion · 07/27/2006 (11:04 am) · 16 replies
Shaders are a hot topic lately, whether its using shaders in TSE or integrating shaders into TGE. Shaders have completely changed the face of realtime graphics by allowing developers to directly control what the GPU does, ushering in a new era of programmable graphics. In fact, modern GPUs no longer support fixed function except through an emulation layer which is composed of numerous highly optimized shaders designed to mimic the fixed function pipeline.
With that in mind, I'm starting another Q&A/FAQ thread similar to the OpenGL thread. This thread is open to any and all questions pertaining to shaders of any kind, and any part of shader development. Whether you'd like to know about the details of integrating HLSL shaders into an application, how GLSL handles shader object management, or the happy details of writing a Cg shader than can compile to the FP20 or ps_1_x profiles, just ask here, and I will do my best to answer your question. If someone else has an answer, that's awesome! Post it! I'll try and keep this file updated with a full set of questions and answers, and will hopefully turn it into a TDN article if it becomes worthwhile as one.
If you have any questions specific to how TSE handles shaders, ask it on the TSE private forums. If you have any questions about integrating shaders into TGE email me and I will point you in the right general direction.
Again, please do not get this thread off topic. Do not turn it into a flame war over which shading language is better. In fact, I will not answer "What is the best shading language?". That is a decision you have to make yourself based on your target audience, what your game requires, what you know, and what you are willing to learn.
With that in mind, I'm starting another Q&A/FAQ thread similar to the OpenGL thread. This thread is open to any and all questions pertaining to shaders of any kind, and any part of shader development. Whether you'd like to know about the details of integrating HLSL shaders into an application, how GLSL handles shader object management, or the happy details of writing a Cg shader than can compile to the FP20 or ps_1_x profiles, just ask here, and I will do my best to answer your question. If someone else has an answer, that's awesome! Post it! I'll try and keep this file updated with a full set of questions and answers, and will hopefully turn it into a TDN article if it becomes worthwhile as one.
If you have any questions specific to how TSE handles shaders, ask it on the TSE private forums. If you have any questions about integrating shaders into TGE email me and I will point you in the right general direction.
Again, please do not get this thread off topic. Do not turn it into a flame war over which shading language is better. In fact, I will not answer "What is the best shading language?". That is a decision you have to make yourself based on your target audience, what your game requires, what you know, and what you are willing to learn.
About the author
#2
A4: Yes.
Vertex Shader:
Fragment Shader:
Q5: Whoa, the GLSL shader looks completely different. What's going on, and why is it so different?
A5: GLSL includes a very large number of built in variables that are set by OpenGL. This variables include the current ModelView matrix, the current projection matrix, the current ModelViewMatrix, the current active lights and their information, etc. In effect, almost any part of the OpenGL render state that could affect the vertex or fragment shader is available to the shader as a built in value, and does not need to be specified by the application.
GLSL also includes a number of built in input and output variables. For example, gl_Vertex contains the vertex position, and gl_Position is the output register for the vertex position in clip space. gl_TexCoord[0] is the output register for a texture coordinate, and gl_MultiTexCoord0 stores the texture coordinates assigned to the first texture unit.
In the fragment shader, gl_FragColor is the output variable to write color to the main render target (usually the main window, though it may be a texture).
One important thing to note is that GLSL does not allow main to return anything, or to accept any inputs.
Another important thing to note is that GLSL does not allow uniforms to be initialized to any values. In HLSL, we said
to set the texture sampler2D to the value stored in TEXUNIT0. In GLSL this must be set directly from the application. That would look something like this:
Fire away! Oh, and if anyone wants to improve the basic shader sample, please do.
07/27/2006 (11:04 am)
Q4: Can you show the same shader as a GLSL shader?A4: Yes.
Vertex Shader:
void main()
{
gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
gl_TexCoord[0] = gl_MultiTexCoord0;
}Fragment Shader:
uniform sampler2D texture;
void main()
{
gl_FragColor = texture2D(texture, gl_TexCoord[0].st);
}Q5: Whoa, the GLSL shader looks completely different. What's going on, and why is it so different?
A5: GLSL includes a very large number of built in variables that are set by OpenGL. This variables include the current ModelView matrix, the current projection matrix, the current ModelViewMatrix, the current active lights and their information, etc. In effect, almost any part of the OpenGL render state that could affect the vertex or fragment shader is available to the shader as a built in value, and does not need to be specified by the application.
GLSL also includes a number of built in input and output variables. For example, gl_Vertex contains the vertex position, and gl_Position is the output register for the vertex position in clip space. gl_TexCoord[0] is the output register for a texture coordinate, and gl_MultiTexCoord0 stores the texture coordinates assigned to the first texture unit.
In the fragment shader, gl_FragColor is the output variable to write color to the main render target (usually the main window, though it may be a texture).
One important thing to note is that GLSL does not allow main to return anything, or to accept any inputs.
Another important thing to note is that GLSL does not allow uniforms to be initialized to any values. In HLSL, we said
uniform sampler2D texture : TEXUNIT0
to set the texture sampler2D to the value stored in TEXUNIT0. In GLSL this must be set directly from the application. That would look something like this:
U32 textureLoc = glGetUniformLocation(myShader, "texture"); glUseProgram(myShader); glUniform1i(textureLoc, 0);
Fire away! Oh, and if anyone wants to improve the basic shader sample, please do.
#3
Are you going to publish your TGE shader framework? Or is it published as a resource already? Or are you going to make it as a pack/kit? I'm a bit confused with the current state of work for shader post-processing in TGE.
07/27/2006 (11:43 am)
Awesome job, Alex!Are you going to publish your TGE shader framework? Or is it published as a resource already? Or are you going to make it as a pack/kit? I'm a bit confused with the current state of work for shader post-processing in TGE.
#4
My question has more to do with what kinds of rendering effects are possible using the shader code. More to the point, can you give us a complete list of effects so artist types can make an overall informed decision on what the final look of their projects will be? For example, in your other post you give samples of a toon render, black and white, sepia and others. Are the samples you give in that post all that's possible with the DRL/GLSL code? Or are there others we haven't seen yet?
07/27/2006 (11:48 am)
Great article, Alex! I'm sure this will be a nice reference for shader users .My question has more to do with what kinds of rendering effects are possible using the shader code. More to the point, can you give us a complete list of effects so artist types can make an overall informed decision on what the final look of their projects will be? For example, in your other post you give samples of a toon render, black and white, sepia and others. Are the samples you give in that post all that's possible with the DRL/GLSL code? Or are there others we haven't seen yet?
#5
@N R Bharathae: Different kinds of rendering effects possible with shaders... oh boy. Let me use a bad analogy (and I'm assuming here that you're an artist).
You can make any model you want, so long as it has less than 1,500 polygons. How many different models can you make?
Shader writing is just like that. Shader programmers get x instructions (anywhere from 16 to 65,536) to do whatever they want. You can do pretty much any effect imaginable. All the examples that are in my latest .plan used less than 10 instructions, and my card supports up to 64. There is a LOT that could be done.
So, informed decision for post processing: Take a screenshot, run it through photoshop filters until you get the desired look. Hire a good shader programmer, tell him/her what filters you used. There is a very good chance that said shader programmer will have a hacked up shader doing roughly what you did in photoshop in realtime within a couple of hours. From there, just refine and optimize the shader as necessary.
07/27/2006 (12:49 pm)
@Alexander: I will not be releasing any shader code for TGE for a variety of reasons that I'd rather not talk about. As said, if you are interested in integrating shaders into TGE, you can email me and I will point you in the right general direction.@N R Bharathae: Different kinds of rendering effects possible with shaders... oh boy. Let me use a bad analogy (and I'm assuming here that you're an artist).
You can make any model you want, so long as it has less than 1,500 polygons. How many different models can you make?
Shader writing is just like that. Shader programmers get x instructions (anywhere from 16 to 65,536) to do whatever they want. You can do pretty much any effect imaginable. All the examples that are in my latest .plan used less than 10 instructions, and my card supports up to 64. There is a LOT that could be done.
So, informed decision for post processing: Take a screenshot, run it through photoshop filters until you get the desired look. Hire a good shader programmer, tell him/her what filters you used. There is a very good chance that said shader programmer will have a hacked up shader doing roughly what you did in photoshop in realtime within a couple of hours. From there, just refine and optimize the shader as necessary.
#6
07/27/2006 (1:46 pm)
OK, good enough. Thanks for the response.
#7
07/27/2006 (4:12 pm)
Thanks for this Alex! I'm starting to try to learn how to write shaders, this has helped alot
#8
07/27/2006 (4:19 pm)
@Mincetro: If you have any questions, ask them here. This is a two way deal. People ask questions, people give answers, and I keep the HTML file updated so it's one nice FAQ. 
#9
07/27/2006 (4:48 pm)
This should really be in TDN, actually. It would be way easier to maintain and update.
#10
Either way, I'd like to keep the threads going, just because a forum thread is a lot more two way than a TDN entry. Discussion of the answers is probably more important than the answers themselves.
Edit: Or maybe I could just read the TDN stuff on where to put things in TDN... ya. That would be a good idea, wouldn't it?
Edit: Is this a good start?
07/27/2006 (6:43 pm)
@Pat: I did say I would turn it into a TDN article eventually. Eventually can be tomorrow I guess. Any suggestions on where to put it though? I'd like it to be as public as possible given that there is no discussion of TSE (or TGE) specifically here. Same goes for the OpenGL thread.Either way, I'd like to keep the threads going, just because a forum thread is a lot more two way than a TDN entry. Discussion of the answers is probably more important than the answers themselves.
Edit: Or maybe I could just read the TDN stuff on where to put things in TDN... ya. That would be a good idea, wouldn't it?
Edit: Is this a good start?
#11
07/27/2006 (7:47 pm)
Many thanks for this compiled set of usefull information, I'll have to try my hand at shaders as soon as I get my new graphics card (I'm running an onboard one right now :P).
#12
my email is charles (at) clacket.com
09/01/2006 (7:39 am)
@alex Can u tell me how to integrate shader in TGE !!!!my email is charles (at) clacket.com
#13
09/05/2006 (3:16 pm)
I have an idea : Add screenshots of examples of the different types of shaders at work. Whether screenshots of Torque or just a the base OpenGL/Direct3D rotating cube code.
#15
09/06/2006 (3:47 pm)
@Randy: Look up a couple of posts. See the link? Here it is again TDN Article 
Associate Alex Scarborough
Q1: What exactly is a shader?
A1: A shader can be best though of as a script that executes on your graphics card. There are currently two main kinds of shaders, vertex shaders, and pixel shaders (fragment shaders to the OpenGL crowd).
Vertex shaders take as their input the vertex position, texture coordinates for that vertex, and other data that may be sent in. As output, they give the vertex position in clip-space and whatever data the pixel shader needs. This could be texture coordinates, a color or two, lighting information, or just about anything else. There are restrictions on how much data the vertex shader can send to the pixel shader which depend on the GPU and also API being used.
Pixel shaders take as their input whatever the vertex shader passes them. As output, they write color data to the current render target(s) which is generally the application window, though it may be a texture, or textures as the case may be.
Q2: Can you show me an example of an HLSL shader?
A2: Yes.
Vertex Shader:
struct AppData { float4 Pos : POSITION; float2 UV0 : TEXCOORD0; }; struct VertexOut { float4 Pos : POSITION; float2 Tex0 : TEXCOORD0; }; VertexOut main(AppData IN, uniform float4x4 ModelViewProjMatrix) { VertexOut OUT; OUT.Pos = mul(IN.Pos, ModelViewProjMatrix); OUT.Tex0 = IN.UV0; return OUT; }Fragment Shader:
struct VertexOut { float4 Pos : POSITION; float2 Tex0 : TEXCOORD0; }; struct PixelOut { float4 Color : COLOR0; }; PixelOut main(VertexOut IN, uniform sampler2D texture : TEXUNIT0) { PixelOut OUT; OUT.Color = tex2D(texture, IN.Tex0); return OUT; }Q3: Can you explain what each line of that shader does?
A3: Yes.
struct AppData { float4 Pos : POSITION; float2 UV0 : TEXCOORD0; };This struct tells us what the application is passing into the vertex shader. In this case, we have a float4 (ie 4 floats in a vector) named Pos which takes the vertex position, and a float2 (ie 2 floats in a vector) named UV0 which takes the first texture coordinate assigned to the vertex.
struct VertexOut { float4 Pos : POSITION; float2 Tex0 : TEXCOORD0; };This struct tells us what the vertex shader is going to output. We obviously must output the vertex position in clip space, and we are also outputting a texture coordinate which will be assigned to TEXCOORD0 when it is passed into the pixel shader.
This defines the main() function in the vertex shaders. All shaders must have a main function. This function returns a VertexOut, and as input takes an AppData, and a uniform float4x4 holding (hopefully) the Modelview-Projection matrix.
A uniform value is a value set by the application. A float4x4 is a 4x4 matrix.
This multiplies our input vertex position by the modelview projection matrix and writes that result to the output register for vertex position in clip space. It also writes the input texture coordinate into the output texture coordinate register without any modification. Finally, it returns OUT, ending the main function, and the vertex shader. On to the pixel shader.
struct VertexOut { float4 Pos : POSITION; float2 Tex0 : TEXCOORD0; };Now defines what the input to the pixel shader is, and what registers this input is located in. Just copying the VertexOut struct was suitable for our needs, and saved a few seconds of typing up another struct.
struct PixelOut { float4 Color : COLOR0; };This struct defines our output from the pixel shader. In this case, we are outputting only one color, most likely to the main window.
This defines the main function for our pixel shader. It returns a PixelOut, and as input takes a uniform sampler2D.
A sampler2D is a variable type which is used in texture lookups. In this case, we are taking the value stored in the TEXUNIT0 register and assigning it to our sampler2D.
This reads the texture defined by the sampler2D texture at the texture coordinates IN.Tex0. It writes the result into the Color member of the PixelOut struct. After this, OUT is returned, ending the pixel shader, and writing the color into the current active render target (probably the main window).