Stencil Shadows in TGE
by Brett Fattori · 03/29/2004 (11:04 am) · 75 comments
Download Code File
I haven't updated this code in over a year. This isn't guaranteed to work with the latest version of HEAD. Sorry.
Examples:
Buggy with shadows
Orc with shadows
More buggy with shadows
Orc close-up
From the read me file:
I'll warn you right now... this is going to be rough. There were a lot of modifications to the engine to get stencil shadows working. It's not going to be a simple drop in, compile and run. There's a ton of code that is added and/or modified in the engine. Additionally, this has only been implemented and tested on the Windows platform.
Included in this archive, you'll find:
+ stencil_032804.patch
+ shadowVolume.cc
+ shadowVolume.h
+ stencil_shadows.txt
+ optionsDlg.gui
+ readMe.txt (This file)
Going the patch route
========================
You can install the patch against the HEAD revision of the TGE. This method should be used by people familiar with using patches. You'll have to troll through the code and fix any problems from conflicts and such. Just run the patch against a fresh HEAD
revision. For example:
cd c:\torque
patch -p0 < stencil_032804.patch
Manual Route
=========================
Inside the file "stencil_shadows.txt" you'll find step-by-step modifications to the engine to add volume stencil shadows. I've attempted to give enough information to know where to put the bits of code. You only need to add/modify the code that is enclosed in the DK_911 tokens. It's going to take a while to go this route...
shadowVolume.cc and shadowVolume.h
==================================
You'll need to extract these files into your "/engine/game/" directory. Make sure you add them to the project file, under the "game" folder of the Torque.
Notes
===============
Z-fail shadows are not working. I tried -- there's some code in there that is the beginnings of Z-fail, but it's incomplete. Z-pass is working, however, quite nicely. You're going to see a framerate drop, unless you have some fast hardware. It's inevitable... I've done some optimizations to the code to try to minimize this. Plus, there are numerous extensions that were implemented to try to combat this. Interior lights do not cause shadows. I didn't get around to implementing these... I wasn't sure where to begin looking for the code that managed interior light sources.
Things to know
===================
There are some script commands for toggling stencil shadows, and setting detail levels. I have included a replacement for the "optionsDlg.gui" file that implements some of these settings.
Console Commands:
-----------------
toggleStencilShadows() - Toggles stencil shadows on/off
toggleStencilZFail() - Toggles use of Z-Fail code
toggleStencilExtTri() - Toggles use of "External Triangle" rendering method
setSilhouetteDelta(F32) - Set a value to determine detail level of object used for silhouette generation
toggleFastStencils() - [NOT IMPLEMENTED ANY MORE]
Associated Prefs:
-----------------
$pref::StencilShadow::renderZFailShadows
$pref::StencilShadow::externalTris
$pref::StencilShadow::infiniteProjection
$pref::StencilShadow::enableStencilShadows
$pref::StencilShadow::detailDelta
$pref::StencilShadow::gFastStencilRendering
Changes to the code
===========================
Please contribute any changes back to the community! This should be an ongoing project so that everyone can enjoy volume shadows in the TGE. For the moment, submit any changes to me at:
brettf AT renderengine DOT com
This is, by no means, a complete implementation (or even correct) of volume stencil shadows. There were numerous changes made during the development of this code that influenced design decisions.
Originally, the shadows we being rendered in a two-pass manner, but a faster one-pass system was chosen to keep speeds up. I expect that there are further optimizations that can be made to this code, and I hope that you'll share what you find and change!
Using the shadows
==========================
People have been asking me what the settings are for the datablock. Plus, it would be helpful to know what they do. These can be added to any object that derives from shapeBase or tsStatic. Here is a short breakdown of the items:
enableStencilShadows = true/false
Enable stencil shadows for this object.
sunShadowsOnly = true/false
If set to true, shadows will only be cast for the sun light (light zero in the light manager) The shadow generator will ignore the next two settings if this is set to true.
pointShadows = true/false
Set this to true to enable shadow casting for point light sources. The point light must be set to cast stencil shadows as well.
vectorShadows = true/false
If this is true, vector lights (other than the sun) will cause this object to cast shadows. Like point lights, the light must be set to cast stencil shadows as well.
alphaDistance true/false
Setting this to true will make the shadow vary in alpha based on distance from the light source.
shadowBaseAlpha = float
This is the amount of alpha the shadow will use. A value of 0.5 is pretty good.
updateEveryFrame = true/false
For shapes that do not animate, there isn't any real need to regen the silhouette for every single frame. Only when the object moves or the light moves. When this is true, the following occurs: if the object has changed position (not rotated, must change X,Y, or Z) or the light has changed position (both by some fixed epsilon) the shadow will update. Otherwise, the shadow's silhouette and volume don't get regenerated. For animated shapes (ones with ambient animations especially) set this to true. The shadow will regen each frame.
highDetail = true/false
Setting this to true will cause shadows to always generate at the highest distance detail for the object. In other words, based on the current LOD for the object. Setting this to false will cause the object's shadow to be generated from the distance detail that is n detail levels less than the current LOD. This means less polys in the shape will be used to generate the silhouette and shadow volume. Check the value in $pref::StencilShadow::detailDelta to see the delta between the highest current LOD and what will be used to generate the shadow. The value in the pref is between 0 and 10. The actual value used is subtracted from 10 to get how many LODs "down" it should use. ie: Setting it to 10 will use the highest (10 - 10 = 0 LODs down) Setting it to 5 will use 5 levels down (if available) (10 - 5 = 5 LODs down).
debugVolumes = true/false
If you want to look at the actual volume cast by your shape (not just the shadow) set this to true.
debugLightLines = true/false
To see which lights are effecting this object, set this to true. A line will be drawn (in the light's color, no less) from the light to the object.
I haven't updated this code in over a year. This isn't guaranteed to work with the latest version of HEAD. Sorry.
Examples:
Buggy with shadows
Orc with shadows
More buggy with shadows
Orc close-up
From the read me file:
I'll warn you right now... this is going to be rough. There were a lot of modifications to the engine to get stencil shadows working. It's not going to be a simple drop in, compile and run. There's a ton of code that is added and/or modified in the engine. Additionally, this has only been implemented and tested on the Windows platform.
Included in this archive, you'll find:
+ stencil_032804.patch
+ shadowVolume.cc
+ shadowVolume.h
+ stencil_shadows.txt
+ optionsDlg.gui
+ readMe.txt (This file)
Going the patch route
========================
You can install the patch against the HEAD revision of the TGE. This method should be used by people familiar with using patches. You'll have to troll through the code and fix any problems from conflicts and such. Just run the patch against a fresh HEAD
revision. For example:
cd c:\torque
patch -p0 < stencil_032804.patch
Manual Route
=========================
Inside the file "stencil_shadows.txt" you'll find step-by-step modifications to the engine to add volume stencil shadows. I've attempted to give enough information to know where to put the bits of code. You only need to add/modify the code that is enclosed in the DK_911 tokens. It's going to take a while to go this route...
shadowVolume.cc and shadowVolume.h
==================================
You'll need to extract these files into your "/engine/game/" directory. Make sure you add them to the project file, under the "game" folder of the Torque.
Notes
===============
Z-fail shadows are not working. I tried -- there's some code in there that is the beginnings of Z-fail, but it's incomplete. Z-pass is working, however, quite nicely. You're going to see a framerate drop, unless you have some fast hardware. It's inevitable... I've done some optimizations to the code to try to minimize this. Plus, there are numerous extensions that were implemented to try to combat this. Interior lights do not cause shadows. I didn't get around to implementing these... I wasn't sure where to begin looking for the code that managed interior light sources.
Things to know
===================
There are some script commands for toggling stencil shadows, and setting detail levels. I have included a replacement for the "optionsDlg.gui" file that implements some of these settings.
Console Commands:
-----------------
toggleStencilShadows() - Toggles stencil shadows on/off
toggleStencilZFail() - Toggles use of Z-Fail code
toggleStencilExtTri() - Toggles use of "External Triangle" rendering method
setSilhouetteDelta(F32) - Set a value to determine detail level of object used for silhouette generation
toggleFastStencils() - [NOT IMPLEMENTED ANY MORE]
Associated Prefs:
-----------------
$pref::StencilShadow::renderZFailShadows
$pref::StencilShadow::externalTris
$pref::StencilShadow::infiniteProjection
$pref::StencilShadow::enableStencilShadows
$pref::StencilShadow::detailDelta
$pref::StencilShadow::gFastStencilRendering
Changes to the code
===========================
Please contribute any changes back to the community! This should be an ongoing project so that everyone can enjoy volume shadows in the TGE. For the moment, submit any changes to me at:
brettf AT renderengine DOT com
This is, by no means, a complete implementation (or even correct) of volume stencil shadows. There were numerous changes made during the development of this code that influenced design decisions.
Originally, the shadows we being rendered in a two-pass manner, but a faster one-pass system was chosen to keep speeds up. I expect that there are further optimizations that can be made to this code, and I hope that you'll share what you find and change!
Using the shadows
==========================
People have been asking me what the settings are for the datablock. Plus, it would be helpful to know what they do. These can be added to any object that derives from shapeBase or tsStatic. Here is a short breakdown of the items:
enableStencilShadows = true/false
Enable stencil shadows for this object.
sunShadowsOnly = true/false
If set to true, shadows will only be cast for the sun light (light zero in the light manager) The shadow generator will ignore the next two settings if this is set to true.
pointShadows = true/false
Set this to true to enable shadow casting for point light sources. The point light must be set to cast stencil shadows as well.
vectorShadows = true/false
If this is true, vector lights (other than the sun) will cause this object to cast shadows. Like point lights, the light must be set to cast stencil shadows as well.
alphaDistance true/false
Setting this to true will make the shadow vary in alpha based on distance from the light source.
shadowBaseAlpha = float
This is the amount of alpha the shadow will use. A value of 0.5 is pretty good.
updateEveryFrame = true/false
For shapes that do not animate, there isn't any real need to regen the silhouette for every single frame. Only when the object moves or the light moves. When this is true, the following occurs: if the object has changed position (not rotated, must change X,Y, or Z) or the light has changed position (both by some fixed epsilon) the shadow will update. Otherwise, the shadow's silhouette and volume don't get regenerated. For animated shapes (ones with ambient animations especially) set this to true. The shadow will regen each frame.
highDetail = true/false
Setting this to true will cause shadows to always generate at the highest distance detail for the object. In other words, based on the current LOD for the object. Setting this to false will cause the object's shadow to be generated from the distance detail that is n detail levels less than the current LOD. This means less polys in the shape will be used to generate the silhouette and shadow volume. Check the value in $pref::StencilShadow::detailDelta to see the delta between the highest current LOD and what will be used to generate the shadow. The value in the pref is between 0 and 10. The actual value used is subtracted from 10 to get how many LODs "down" it should use. ie: Setting it to 10 will use the highest (10 - 10 = 0 LODs down) Setting it to 5 will use 5 levels down (if available) (10 - 5 = 5 LODs down).
debugVolumes = true/false
If you want to look at the actual volume cast by your shape (not just the shadow) set this to true.
debugLightLines = true/false
To see which lights are effecting this object, set this to true. A line will be drawn (in the light's color, no less) from the light to the object.
About the author
Recent Blogs
• A small hello• Recent Stuph
• So long and so out of touch...
• Plan for Brett Fattori
• Plan for Brett Fattori
#42
04/10/2004 (7:28 pm)
@ Simon "not sure what sort of problem you
#43
thanks in advance.
04/11/2004 (11:06 am)
i am having issues with no shadows showing up, the player puts out no shadow, the buildings etc still put off their standard. i have done a manual implementation, over a moded 1.2. the manual file does not say what do do other than code, i have pieced together a few things from the comments, but can i get a step by step on what to do after the code is in and working.thanks in advance.
#44
Make sure you've "turned them on". Open the console and type in:
04/15/2004 (8:08 am)
@SynditechMake sure you've "turned them on". Open the console and type in:
toggleStencilShadows()
#46
so i alternate between old shadows and no shadows at all.
the code is all in, and i have put the scripts in that a saw in the patch file.
ive spent a fair amount of time on this but i seem to be stuck.
any ideas?
04/18/2004 (9:58 pm)
i am using a modified "test" mission and i cant get any shadows at all on the player, the toggle shadows only put the old shadows on,so i alternate between old shadows and no shadows at all.
the code is all in, and i have put the scripts in that a saw in the patch file.
ive spent a fair amount of time on this but i seem to be stuck.
any ideas?
#47
04/19/2004 (12:19 am)
I have the same result Synditech, Ive done it manually, and gone through .patch file again to make sur ei ddint miss anything.
#48
anything you think we could have missed.
04/19/2004 (3:40 am)
John can you tell me what stages you went through to get this working, you would have used the manual implementation and then pulled hte patch apart to finish it?anything you think we could have missed.
#49
04/21/2004 (7:28 pm)
has anyone else had the issue of the shadows just not comming up?
#50
Shapes using alpha transparency in pngs get this error.
Also i've noticed that when you stand infront of an object that cast a shadow, aka you get part of the shadow on you, if you move the camera around abit suddenly the shadows get inverted, you get light where the shadow is, and the rest of the world gets shadow.
Is there a way to apply Stencil Shadows to the fxReplicators? Especially fxShapeRelicator.
Lars
04/28/2004 (3:25 pm)
Click here for screenshotShapes using alpha transparency in pngs get this error.
Also i've noticed that when you stand infront of an object that cast a shadow, aka you get part of the shadow on you, if you move the camera around abit suddenly the shadows get inverted, you get light where the shadow is, and the rest of the world gets shadow.
Is there a way to apply Stencil Shadows to the fxReplicators? Especially fxShapeRelicator.
Lars
#51
05/02/2004 (1:21 am)
I just ran the patch on a fresh HEAD as of 5/1/04 at around 11pm. The patch had two failures, which I went back and cleaned up. Everything compiled fine. However it gives me an exception error whenever stencil shadows are enabled . . . the error will occur at the end of the mission load if I have stencils enabled before loading, and it will crash at the first screen refresh if I enable stencils during a mission in session. Has there been a change to HEAD that breaks this resource or am I just getting bad voo doo from my machine?
#52
@Robert: Haven't had time to work this out with the latest HEAD release. It might be that there are too many changes to the code for this resource to work properly. However, if you posted your errors, I might be able to help with the solution.
- Brett
05/03/2004 (7:07 am)
@Lars: That's what I'd expect from alpha maps. Unfortunately, there isn't an easy way to generate stenciled shadows for alpha maps. They're just quads with textures on them, so the geometry doesn't exist to generate silhouettes. As for applying stencils to fxReplicators, you'd have to open that code up and make sure that the datablock has stencil shadows enabled for each newly created object. The fxShapeReplicator should be the simplest, as it's just duplicating the datablock (which is where you specify stencil shadows on/off for an object)@Robert: Haven't had time to work this out with the latest HEAD release. It might be that there are too many changes to the code for this resource to work properly. However, if you posted your errors, I might be able to help with the solution.
- Brett
#53
patch the head
it'll fail on shapebase, so do that part by hand.
pay attention to the code you add. i may have just ignored the rejects.
compile and stencil shadows!
then to fix the crash:
array out of bounds around p .. not sure what else this'll break .. but it works now :)
05/05/2004 (5:11 pm)
heres what i did to get this working.patch the head
it'll fail on shapebase, so do that part by hand.
pay attention to the code you add. i may have just ignored the rejects.
compile and stencil shadows!
then to fix the crash:
array out of bounds around p .. not sure what else this'll break .. but it works now :)
in shadowvolume.cc
in bool StencilShadowInstance::setClipBounds(const Point3F eye)
{
RectI vp;
S32 scissor[4];
Point3F p[8]; //changed this from Point3F p[7];
...like i said not sure what all this'll break.. :)
#54
05/10/2004 (8:34 pm)
#55
I just got the resource to compile against HEAD but I ran into an issue. Each time I start the mission it gets the the second screen and fails with Out of Range Read.
Here is the last few lines of my console.log
Could not locate texture: starter.fps/data/shapes/player/crossbow
Could not locate texture: starter.fps/data/shapes/player/clip
Validation required for shape: starter.fps/data/shapes/player/player.dts
Could not locate texture: starter.fps/data/shapes/player/base.lmale
Could not locate texture: starter.fps/data/shapes/player/base.lmale
Could not locate texture: starter.fps/data/shapes/player/crossbow
Could not locate texture: starter.fps/data/shapes/player/clip
Validation required for shape: starter.fps/data/shapes/player/player.dts
Mapping string: MissionStartPhase2 to index: 9
*** Phase 2: Download Ghost Objects
Mapping string: MissionStartPhase2Ack to index: 1
Warning: (e:\code\torque\engine\core\bitstream.cc @ 258) Out of range read
Warning: (e:\code\torque\engine\core\bitstream.cc @ 258) Out of range read
Warning: (e:\code\torque\engine\core\bitstream.cc @ 258) Out of range read
Warning: (e:\code\torque\engine\core\bitstream.cc @ 258) Out of range read
Does anybody know how I would go about debugging this. I tried a debugger but it doesn't lead me back to what made the call. Also I tried a cvs diff to see if I could seen misconfigured but I was unable to find anything.
VS.Net 2003 on winxp.
Thanks, Ben
06/23/2004 (12:05 pm)
Hello Everybody, I just got the resource to compile against HEAD but I ran into an issue. Each time I start the mission it gets the the second screen and fails with Out of Range Read.
Here is the last few lines of my console.log
Could not locate texture: starter.fps/data/shapes/player/crossbow
Could not locate texture: starter.fps/data/shapes/player/clip
Validation required for shape: starter.fps/data/shapes/player/player.dts
Could not locate texture: starter.fps/data/shapes/player/base.lmale
Could not locate texture: starter.fps/data/shapes/player/base.lmale
Could not locate texture: starter.fps/data/shapes/player/crossbow
Could not locate texture: starter.fps/data/shapes/player/clip
Validation required for shape: starter.fps/data/shapes/player/player.dts
Mapping string: MissionStartPhase2 to index: 9
*** Phase 2: Download Ghost Objects
Mapping string: MissionStartPhase2Ack to index: 1
Warning: (e:\code\torque\engine\core\bitstream.cc @ 258) Out of range read
Warning: (e:\code\torque\engine\core\bitstream.cc @ 258) Out of range read
Warning: (e:\code\torque\engine\core\bitstream.cc @ 258) Out of range read
Warning: (e:\code\torque\engine\core\bitstream.cc @ 258) Out of range read
Does anybody know how I would go about debugging this. I tried a debugger but it doesn't lead me back to what made the call. Also I tried a cvs diff to see if I could seen misconfigured but I was unable to find anything.
VS.Net 2003 on winxp.
Thanks, Ben
#56
http://www.benwoodhead.com/torque/stencilpatch.zip
It doesn't work but I would love some help getting it to work against the current head.
Thanks, Ben
06/24/2004 (4:58 am)
Here is a link to this resource against the head. http://www.benwoodhead.com/torque/stencilpatch.zip
It doesn't work but I would love some help getting it to work against the current head.
Thanks, Ben
#57
http://www.benwoodhead.com/torque/stencilpatch.zip
Ben
06/24/2004 (6:22 am)
I followed GG instructions for creating the patch so it should apply cleanly. http://www.benwoodhead.com/torque/stencilpatch.zip
Ben
#58
Its fixed :). After patching todays head with the http://www.benwoodhead.com/torque/stencilpatch.zip please go to game/TSStatic.cc and find this method.
You will find that this line:
Is at bottom of the shadow stuff and should be above the shadow stuff.
Like this:
Ben
06/24/2004 (11:30 am)
Hello, Its fixed :). After patching todays head with the http://www.benwoodhead.com/torque/stencilpatch.zip please go to game/TSStatic.cc and find this method.
void TSStatic::unpackUpdate(NetConnection *con, BitStream *stream)
You will find that this line:
mShapeName = stream->readSTString();
Is at bottom of the shadow stuff and should be above the shadow stuff.
Like this:
void TSStatic::unpackUpdate(NetConnection *con, BitStream *stream)
{
Parent::unpackUpdate(con, stream);
MatrixF mat;
Point3F scale;
mathRead(*stream, &mat);
mathRead(*stream, &scale);
setScale(scale);
setTransform(mat);
mShapeName = stream->readSTString();
mStencilShadowCaster = stream->readFlag();
mDebugVolumes = stream->readFlag();
mShadowSunlightOnly = stream->readFlag();
mShadowPointLights = stream->readFlag();
mShadowVectorLights = stream->readFlag();
mDistanceAffectsAlpha = stream->readFlag();
stream->read(&mShadowBaseAlpha);
mDebugLightLines = stream->readFlag();
mUpdateEveryFrame = stream->readFlag();
mUseHighestDetail = stream->readFlag();
}Ben
#59
Although it should work, I have a nvidia geforce 5 256mb - if that is any help...
06/25/2004 (11:24 am)
I have everything compiling correctly and the levels load, but say when I load the tutorial.base and make a static mesh and turn stencil shadows on the game crash's... when I load the starter.fps level it crash's as well, I think it is having a problem with the rendering...Although it should work, I have a nvidia geforce 5 256mb - if that is any help...
#60
Did you follow the last instruction I gave. I had a cut and paste error :). Check the TSStatic.cc file and make sure its packing stuff is correct. Follow the last post from me.
Just incase that is not the problem, what error are you getting.
Ben
06/25/2004 (12:00 pm)
Hello Josh, Did you follow the last instruction I gave. I had a cut and paste error :). Check the TSStatic.cc file and make sure its packing stuff is correct. Follow the last post from me.
Just incase that is not the problem, what error are you getting.
Ben
Torque Owner Simon Windmill
- The stencil shadows don't render when you move into the shadow of a building. This is because the shadow volume generation code checks to see if terrain or interiors are occluding. You can change this behaviour in shadowVolume.cc, StencilShadow::generateShadows, look for the comment // Check if the light is occluded by either of Terrain Interiors
- The alpha stuff not working. Presumably this is because of the change to a single render pass method rather than multiple passes - the shadows get rendered via a single quad, so all shadows must have the same alpha value, the value that John mentions above
I've not played much, and it might be fixable by tweaking the generateShadows function as I mentioned above, but I notice the shadows don't always get rendered onto interiors objects. I wonder if we can fix that..
Bob - not sure what sort of problem you're having, all I did was apply the patch to my modded directory (which was originally a copy of starter.fps), copied the shadowVolume.* files to the right place, and made my project include them.