Stencil Shadows in TGE
by Brett Fattori · 03/29/2004 (11:04 am) · 75 comments
Download Code File
I haven't updated this code in over a year. This isn't guaranteed to work with the latest version of HEAD. Sorry.
Examples:
Buggy with shadows
Orc with shadows
More buggy with shadows
Orc close-up
From the read me file:
I'll warn you right now... this is going to be rough. There were a lot of modifications to the engine to get stencil shadows working. It's not going to be a simple drop in, compile and run. There's a ton of code that is added and/or modified in the engine. Additionally, this has only been implemented and tested on the Windows platform.
Included in this archive, you'll find:
+ stencil_032804.patch
+ shadowVolume.cc
+ shadowVolume.h
+ stencil_shadows.txt
+ optionsDlg.gui
+ readMe.txt (This file)
Going the patch route
========================
You can install the patch against the HEAD revision of the TGE. This method should be used by people familiar with using patches. You'll have to troll through the code and fix any problems from conflicts and such. Just run the patch against a fresh HEAD
revision. For example:
cd c:\torque
patch -p0 < stencil_032804.patch
Manual Route
=========================
Inside the file "stencil_shadows.txt" you'll find step-by-step modifications to the engine to add volume stencil shadows. I've attempted to give enough information to know where to put the bits of code. You only need to add/modify the code that is enclosed in the DK_911 tokens. It's going to take a while to go this route...
shadowVolume.cc and shadowVolume.h
==================================
You'll need to extract these files into your "/engine/game/" directory. Make sure you add them to the project file, under the "game" folder of the Torque.
Notes
===============
Z-fail shadows are not working. I tried -- there's some code in there that is the beginnings of Z-fail, but it's incomplete. Z-pass is working, however, quite nicely. You're going to see a framerate drop, unless you have some fast hardware. It's inevitable... I've done some optimizations to the code to try to minimize this. Plus, there are numerous extensions that were implemented to try to combat this. Interior lights do not cause shadows. I didn't get around to implementing these... I wasn't sure where to begin looking for the code that managed interior light sources.
Things to know
===================
There are some script commands for toggling stencil shadows, and setting detail levels. I have included a replacement for the "optionsDlg.gui" file that implements some of these settings.
Console Commands:
-----------------
toggleStencilShadows() - Toggles stencil shadows on/off
toggleStencilZFail() - Toggles use of Z-Fail code
toggleStencilExtTri() - Toggles use of "External Triangle" rendering method
setSilhouetteDelta(F32) - Set a value to determine detail level of object used for silhouette generation
toggleFastStencils() - [NOT IMPLEMENTED ANY MORE]
Associated Prefs:
-----------------
$pref::StencilShadow::renderZFailShadows
$pref::StencilShadow::externalTris
$pref::StencilShadow::infiniteProjection
$pref::StencilShadow::enableStencilShadows
$pref::StencilShadow::detailDelta
$pref::StencilShadow::gFastStencilRendering
Changes to the code
===========================
Please contribute any changes back to the community! This should be an ongoing project so that everyone can enjoy volume shadows in the TGE. For the moment, submit any changes to me at:
brettf AT renderengine DOT com
This is, by no means, a complete implementation (or even correct) of volume stencil shadows. There were numerous changes made during the development of this code that influenced design decisions.
Originally, the shadows we being rendered in a two-pass manner, but a faster one-pass system was chosen to keep speeds up. I expect that there are further optimizations that can be made to this code, and I hope that you'll share what you find and change!
Using the shadows
==========================
People have been asking me what the settings are for the datablock. Plus, it would be helpful to know what they do. These can be added to any object that derives from shapeBase or tsStatic. Here is a short breakdown of the items:
enableStencilShadows = true/false
Enable stencil shadows for this object.
sunShadowsOnly = true/false
If set to true, shadows will only be cast for the sun light (light zero in the light manager) The shadow generator will ignore the next two settings if this is set to true.
pointShadows = true/false
Set this to true to enable shadow casting for point light sources. The point light must be set to cast stencil shadows as well.
vectorShadows = true/false
If this is true, vector lights (other than the sun) will cause this object to cast shadows. Like point lights, the light must be set to cast stencil shadows as well.
alphaDistance true/false
Setting this to true will make the shadow vary in alpha based on distance from the light source.
shadowBaseAlpha = float
This is the amount of alpha the shadow will use. A value of 0.5 is pretty good.
updateEveryFrame = true/false
For shapes that do not animate, there isn't any real need to regen the silhouette for every single frame. Only when the object moves or the light moves. When this is true, the following occurs: if the object has changed position (not rotated, must change X,Y, or Z) or the light has changed position (both by some fixed epsilon) the shadow will update. Otherwise, the shadow's silhouette and volume don't get regenerated. For animated shapes (ones with ambient animations especially) set this to true. The shadow will regen each frame.
highDetail = true/false
Setting this to true will cause shadows to always generate at the highest distance detail for the object. In other words, based on the current LOD for the object. Setting this to false will cause the object's shadow to be generated from the distance detail that is n detail levels less than the current LOD. This means less polys in the shape will be used to generate the silhouette and shadow volume. Check the value in $pref::StencilShadow::detailDelta to see the delta between the highest current LOD and what will be used to generate the shadow. The value in the pref is between 0 and 10. The actual value used is subtracted from 10 to get how many LODs "down" it should use. ie: Setting it to 10 will use the highest (10 - 10 = 0 LODs down) Setting it to 5 will use 5 levels down (if available) (10 - 5 = 5 LODs down).
debugVolumes = true/false
If you want to look at the actual volume cast by your shape (not just the shadow) set this to true.
debugLightLines = true/false
To see which lights are effecting this object, set this to true. A line will be drawn (in the light's color, no less) from the light to the object.
I haven't updated this code in over a year. This isn't guaranteed to work with the latest version of HEAD. Sorry.
Examples:
Buggy with shadows
Orc with shadows
More buggy with shadows
Orc close-up
From the read me file:
I'll warn you right now... this is going to be rough. There were a lot of modifications to the engine to get stencil shadows working. It's not going to be a simple drop in, compile and run. There's a ton of code that is added and/or modified in the engine. Additionally, this has only been implemented and tested on the Windows platform.
Included in this archive, you'll find:
+ stencil_032804.patch
+ shadowVolume.cc
+ shadowVolume.h
+ stencil_shadows.txt
+ optionsDlg.gui
+ readMe.txt (This file)
Going the patch route
========================
You can install the patch against the HEAD revision of the TGE. This method should be used by people familiar with using patches. You'll have to troll through the code and fix any problems from conflicts and such. Just run the patch against a fresh HEAD
revision. For example:
cd c:\torque
patch -p0 < stencil_032804.patch
Manual Route
=========================
Inside the file "stencil_shadows.txt" you'll find step-by-step modifications to the engine to add volume stencil shadows. I've attempted to give enough information to know where to put the bits of code. You only need to add/modify the code that is enclosed in the DK_911 tokens. It's going to take a while to go this route...
shadowVolume.cc and shadowVolume.h
==================================
You'll need to extract these files into your "/engine/game/" directory. Make sure you add them to the project file, under the "game" folder of the Torque.
Notes
===============
Z-fail shadows are not working. I tried -- there's some code in there that is the beginnings of Z-fail, but it's incomplete. Z-pass is working, however, quite nicely. You're going to see a framerate drop, unless you have some fast hardware. It's inevitable... I've done some optimizations to the code to try to minimize this. Plus, there are numerous extensions that were implemented to try to combat this. Interior lights do not cause shadows. I didn't get around to implementing these... I wasn't sure where to begin looking for the code that managed interior light sources.
Things to know
===================
There are some script commands for toggling stencil shadows, and setting detail levels. I have included a replacement for the "optionsDlg.gui" file that implements some of these settings.
Console Commands:
-----------------
toggleStencilShadows() - Toggles stencil shadows on/off
toggleStencilZFail() - Toggles use of Z-Fail code
toggleStencilExtTri() - Toggles use of "External Triangle" rendering method
setSilhouetteDelta(F32) - Set a value to determine detail level of object used for silhouette generation
toggleFastStencils() - [NOT IMPLEMENTED ANY MORE]
Associated Prefs:
-----------------
$pref::StencilShadow::renderZFailShadows
$pref::StencilShadow::externalTris
$pref::StencilShadow::infiniteProjection
$pref::StencilShadow::enableStencilShadows
$pref::StencilShadow::detailDelta
$pref::StencilShadow::gFastStencilRendering
Changes to the code
===========================
Please contribute any changes back to the community! This should be an ongoing project so that everyone can enjoy volume shadows in the TGE. For the moment, submit any changes to me at:
brettf AT renderengine DOT com
This is, by no means, a complete implementation (or even correct) of volume stencil shadows. There were numerous changes made during the development of this code that influenced design decisions.
Originally, the shadows we being rendered in a two-pass manner, but a faster one-pass system was chosen to keep speeds up. I expect that there are further optimizations that can be made to this code, and I hope that you'll share what you find and change!
Using the shadows
==========================
People have been asking me what the settings are for the datablock. Plus, it would be helpful to know what they do. These can be added to any object that derives from shapeBase or tsStatic. Here is a short breakdown of the items:
enableStencilShadows = true/false
Enable stencil shadows for this object.
sunShadowsOnly = true/false
If set to true, shadows will only be cast for the sun light (light zero in the light manager) The shadow generator will ignore the next two settings if this is set to true.
pointShadows = true/false
Set this to true to enable shadow casting for point light sources. The point light must be set to cast stencil shadows as well.
vectorShadows = true/false
If this is true, vector lights (other than the sun) will cause this object to cast shadows. Like point lights, the light must be set to cast stencil shadows as well.
alphaDistance true/false
Setting this to true will make the shadow vary in alpha based on distance from the light source.
shadowBaseAlpha = float
This is the amount of alpha the shadow will use. A value of 0.5 is pretty good.
updateEveryFrame = true/false
For shapes that do not animate, there isn't any real need to regen the silhouette for every single frame. Only when the object moves or the light moves. When this is true, the following occurs: if the object has changed position (not rotated, must change X,Y, or Z) or the light has changed position (both by some fixed epsilon) the shadow will update. Otherwise, the shadow's silhouette and volume don't get regenerated. For animated shapes (ones with ambient animations especially) set this to true. The shadow will regen each frame.
highDetail = true/false
Setting this to true will cause shadows to always generate at the highest distance detail for the object. In other words, based on the current LOD for the object. Setting this to false will cause the object's shadow to be generated from the distance detail that is n detail levels less than the current LOD. This means less polys in the shape will be used to generate the silhouette and shadow volume. Check the value in $pref::StencilShadow::detailDelta to see the delta between the highest current LOD and what will be used to generate the shadow. The value in the pref is between 0 and 10. The actual value used is subtracted from 10 to get how many LODs "down" it should use. ie: Setting it to 10 will use the highest (10 - 10 = 0 LODs down) Setting it to 5 will use 5 levels down (if available) (10 - 5 = 5 LODs down).
debugVolumes = true/false
If you want to look at the actual volume cast by your shape (not just the shadow) set this to true.
debugLightLines = true/false
To see which lights are effecting this object, set this to true. A line will be drawn (in the light's color, no less) from the light to the object.
About the author
Recent Blogs
• A small hello• Recent Stuph
• So long and so out of touch...
• Plan for Brett Fattori
• Plan for Brett Fattori
#2
03/29/2004 (12:43 pm)
Looks awsome. I will definently have to take a look at this. 
#3
03/29/2004 (1:28 pm)
So would this allow a DTS to cast a shadow onto another DTS? Or do the shadows only get cast from DTS onto the terrain and interiors?
#4
Buggy with shadows
Orc with shadows
The performance hit will really be up to you, and your implementation of these. It's not tiny, but it also isn't a massive hit. It will really be a matter of the supporting hardware and the detail level for the shadows.
As for shadow casting on objects, the beauty of stenciled shadows is that they cast onto anything and cause self-shadowing which is something other techniques don't permit.
- Brett
03/29/2004 (1:33 pm)
Here are some videos of stencil shadows in action:Buggy with shadows
Orc with shadows
The performance hit will really be up to you, and your implementation of these. It's not tiny, but it also isn't a massive hit. It will really be a matter of the supporting hardware and the detail level for the shadows.
As for shadow casting on objects, the beauty of stenciled shadows is that they cast onto anything and cause self-shadowing which is something other techniques don't permit.
- Brett
#5
A separate question though, how did you make these videos in such a small footprint? i.e. what tools did you use?
Thanks!
03/29/2004 (3:37 pm)
Wicked. Very nice job and something that will definitely make it into my work. A separate question though, how did you make these videos in such a small footprint? i.e. what tools did you use?
Thanks!
#7
03/29/2004 (5:49 pm)
there is something odd in the buggy video, when the buggy jumps you can see a line under the buggy connecting to the shadow.. not very realistic, is this a designed feature or a bug?
#8
03/29/2004 (6:10 pm)
This looks like a great resouce, I'll let you know how it runs when I get it working =)
#9
- Brett
03/29/2004 (7:23 pm)
Ron, it's a bug in the edge detection code. You have good eyes... ;-) It's probably a non-closed manifold. So, the edge can't be detected at those tris. It's going to take some work to make a model that works every time with the edge detector. But, by all means -- grab the code and get it fixed! I'd love to not have to worry if my model is built correctly..- Brett
#10
03/29/2004 (8:52 pm)
Very nice. Don't know what GG has in mind for shadows in the TSE, but if they haven't done anything yet they might consider using parts of your code...
#11
Does anyone else see the funky orc in the starter.FPS? There's a fourth orc in the middle of the "town" that doesn't animate the model, and also displays a red wireframe version of the model in the same location, along with green polygonal trails that show where the shadow is cast from the model (similar to the video where they are turned on in the editor, only they weren't turned on at the time) all the other models work correctly.
03/30/2004 (12:43 am)
Cool. I got it to compile. Using a primarily command line compiler is a little new to me. Didn't know where to type in the names of the 2 shadowVolume files to get them added to the project until now. Does anyone else see the funky orc in the starter.FPS? There's a fourth orc in the middle of the "town" that doesn't animate the model, and also displays a red wireframe version of the model in the same location, along with green polygonal trails that show where the shadow is cast from the model (similar to the video where they are turned on in the editor, only they weren't turned on at the time) all the other models work correctly.
#12
In shapeBase.h you need to specify
class StencilShadow;
and also it doesnt recognise edgList as an identifier? What class does it belong to? I tried SVEdgeList but it says thats not a defined class.
03/30/2004 (9:00 am)
Ok 2 problems.In shapeBase.h you need to specify
class StencilShadow;
and also it doesnt recognise edgList as an identifier? What class does it belong to? I tried SVEdgeList but it says thats not a defined class.
#13
Good catch Westy! I went back through my code and found a few missing references. I've added them to the stencil_shadows.txt file in the zip and re-uploaded. I've also included a link (above) to a mirror of the zip file on my site.
As for the edgeList, it goes in tsMesh.h around line 128:
- Brett
03/30/2004 (4:49 pm)
You need to add "shadowVolume.cc" and "shadowVolume.h" into the "engine/game/" sub-directory. For MinGW, I'd assume that you'd have to add them to a make file? I don't use it so I don't know for sure.Good catch Westy! I went back through my code and found a few missing references. I've added them to the stencil_shadows.txt file in the zip and re-uploaded. I've also included a link (above) to a mirror of the zip file on my site.
As for the edgeList, it goes in tsMesh.h around line 128:
ToolVector<U16> mergeIndices; ///< the last so many verts merge with these
///< verts to form the next detail level
///< NOT IMPLEMENTED YET
// DK_911 ------------------------------------------------
SVEdgeList * edgeList; // For stenciled shadows
// DK_911 ------------------------------------------------
/// billboard data
Point3F billboardAxis;- Brett
#14
1) fpA.whichSide(v)
In shadowVolume.cc it doesnt find fpA when defined as
PlaneF fpA(t1v1, t1v2, t1v3);
2)glActiveStencilFaceEXT(GL_BACK); undeclared
3)glDepthBoundsEXT(znear, zfar); undeclared
03/30/2004 (11:20 pm)
Ok 3 errors left, all in shadowVolume.cc.1) fpA.whichSide(v)
In shadowVolume.cc it doesnt find fpA when defined as
PlaneF fpA(t1v1, t1v2, t1v3);
2)glActiveStencilFaceEXT(GL_BACK); undeclared
3)glDepthBoundsEXT(znear, zfar); undeclared
#15
Doesn't find fpA? In debug you mean "no plane possible"? Or do you mean fpA isn't defined? I haven't had this particular problem, except when a model is just totally wack.
2) & 3)
Did you grab the latest file from the link above? I noticed that I was missing some additional changes in files.
This should be the complete list of changes. There's some stuff that was missing in in tsStatic.h, fxLight.cc & .h, gl_func.h, winGL.cc and platformGL.cc. The identifier (DK_911) was wrong, I had put "DK_91". Whoops.
- Brett
03/31/2004 (5:16 am)
1)Doesn't find fpA? In debug you mean "no plane possible"? Or do you mean fpA isn't defined? I haven't had this particular problem, except when a model is just totally wack.
2) & 3)
Did you grab the latest file from the link above? I noticed that I was missing some additional changes in files.
This should be the complete list of changes. There's some stuff that was missing in in tsStatic.h, fxLight.cc & .h, gl_func.h, winGL.cc and platformGL.cc. The identifier (DK_911) was wrong, I had put "DK_91". Whoops.
- Brett
#16
2) I have made the new changes, added those new gl definitions, but they are not the same.
03/31/2004 (6:18 am)
1) Yes it says fpA isnt defined left of whichside is undefined identifier.2) I have made the new changes, added those new gl definitions, but they are not the same.
#17
in winGL.cc
and
So why does it say they are undefined?
More on fpA.whichSide error.
'whichSide' : cannot convert parameter 1 from 'class Point4F' to 'const class Point3F &'
Reason: cannot convert from 'class Point4F' to 'const class Point3F'
No constructor could take the source type, or constructor overload resolution was ambiguous
Its when you are using
you sure this is right?
Any ideas?
03/31/2004 (6:52 am)
hmm strangein winGL.cc
glActiveStencilFaceEXT_t glActiveStencilFaceEXT;
and
glDepthBoundsEXT_t glDepthBoundsEXT;
So why does it say they are undefined?
More on fpA.whichSide error.
'whichSide' : cannot convert parameter 1 from 'class Point4F' to 'const class Point3F &'
Reason: cannot convert from 'class Point4F' to 'const class Point3F'
No constructor could take the source type, or constructor overload resolution was ambiguous
Its when you are using
PlaneF varible(parameter1,parameter2,parameter3);
you sure this is right?
Any ideas?
#18
For glActiveStencilFaceEXT and glDepthBoundsEXT make sure you're pointing at gl_funcs.h in the win32 directory. Mine were pointing at the ones in X86Unix. (wierd)
Until I get home from work, here is what you need to add for whichSide:
in /engine/math/mPlane.h
I'm really sorry to everyone about this. I walked through this to make sure it would work and I just plain pooched it. I'll keep the resource as up to date as I can.
- Brett
03/31/2004 (8:00 am)
Man, I thought I had everything in the resource... :-(For glActiveStencilFaceEXT and glDepthBoundsEXT make sure you're pointing at gl_funcs.h in the win32 directory. Mine were pointing at the ones in X86Unix. (wierd)
Until I get home from work, here is what you need to add for whichSide:
in /engine/math/mPlane.h
// DK_911 -------------------------------------------- F32 distToPlane( const Point4F& cp ) const; // DK_911 --------------------------------------------and
// DK_911 --------------------------------------- Side whichSide(const Point4F& cp) const; // DK_911 ---------------------------------------finally
//DK_911 --------------------------------------------
inline F32 PlaneF::distToPlane( const Point4F& cp ) const
{
// return mDot(*this,cp) + d;
return (x * cp.x + y * cp.y + z * cp.z + d * cp.w);
}
inline PlaneF::Side PlaneF::whichSide(const Point4F& cp) const
{
F32 dist = distToPlane(cp);
if (dist >= 0.005f) // if (mFabs(dist) < 0.005f)
return Front; // return On;
else if (dist <= -0.005f) // else if (dist > 0.0f)
return Back; // return Front;
else // else
return On; // return Back;
}
//DK_911 --------------------------------------------I'm really sorry to everyone about this. I walked through this to make sure it would work and I just plain pooched it. I'll keep the resource as up to date as I can.
- Brett
#19
There is a an error after adding whichSide() and distToPlane() because they are laready defined passing a Point3F, so it gives a redefinition error.
Also I still cant see why i get undefined for those two gl statments.
03/31/2004 (11:41 am)
My posts arent getting through :(There is a an error after adding whichSide() and distToPlane() because they are laready defined passing a Point3F, so it gives a redefinition error.
Also I still cant see why i get undefined for those two gl statments.
#20
Check to make sure that those two GL functions are defined properly in the header file that is being used. They should appear in ALL versions of gl_func.h and gl_types.h. I'll be looking at the resource files again so I can get everything in there. In the meantime, check the patch file against what is in the stencil_shadows.txt file. That will give you something to compare to. The patch is just a text file. Look in the platformGL.h file in your platformWin32 directory. You'll notice that it points to the "gl_func.h" file in the platformX86Unix directory. You can either change this to point at the win32 directory or make the changes in the gl_func.h in the X86Unix directory.
- Brett
03/31/2004 (5:58 pm)
If you put whichSide() and distToPlane() into the header file with Point4F you're doing what is called overloading. Thus, you have two definitions of these functions so it will compile and use the correct one for the parameter being passed in. As long as you don't define two functions with the exact same parameters, that is.Check to make sure that those two GL functions are defined properly in the header file that is being used. They should appear in ALL versions of gl_func.h and gl_types.h. I'll be looking at the resource files again so I can get everything in there. In the meantime, check the patch file against what is in the stencil_shadows.txt file. That will give you something to compare to. The patch is just a text file. Look in the platformGL.h file in your platformWin32 directory. You'll notice that it points to the "gl_func.h" file in the platformX86Unix directory. You can either change this to point at the win32 directory or make the changes in the gl_func.h in the X86Unix directory.
- Brett
Torque Owner Thomas \"Man of Ice\" Lund