Beta 2: [Fix] Terrain shadow rendering glitch

#1 I saw somewhere, a way to set these things automaticaly, depending on what you set your terrain size to.
soif you set to 256X256, it will automaticaly set the shadow size, I'll try find it.

#2 The TerrCell::smMinCellSize = 64; and related changes in the terrain files fix it so terrain of 128x128 to import correctly.
Also Beta1 terrain LOD oddity seem remedied by TerrCell::smMinCellSize = 64; and the following place were hard coded 128 was replaced with variable smMinCellSize, not to mention terrain is now rendering much faster.

I think you might be onto something; but it is not necessarily smMinCellSize at fault but one of the other place where it is now being used as a variable that it was not in Beta1. This of course is conjecture, and if i have time i will test it for validity.

#3 Deepscratch.. that would be the solution I guess, although I am worried about
"it also introduced some bizarre geometry errors"

Henry, can you elaborate on that ?

#4 I'm not sure why changing TerrCell::smMinCellSize from 64 back to 128 make the 256x256 terrain shadow look better, but this isn't the right fix.

#5 Yeah, I didn't intend this to be an actual solution, just a bit of temporary help for anyone (like me) who needs to produce screenshots and demo videos right now. :P

In theory this should give some some clues about what's causing the terrain shadow rendering issues. I can't say I know that code well enough to make any guesses, but I did notice pretty consistently that shadow rendering started to behave erratically whenever the terrain consisted of more than 4 cells (ie, cellSize 64- at 256, 128- at 512, etc).

Hewster: The geometry errors I'm referring to occurred when I changed the value to 256. I had a theory about the 512 res terrain, and it actually ended up being right; with only 4 terrain cells, the 512 terrain also rendered shadows correctly, however the engine didn't appreciate these 256x256 bit cells, as the terrain system was clearly never designed to handle cells of this size.

#6 Found another the next link in the chain. That is to say, another change that fixes the terrain shadows.

This one seems to be terrain resolution-independent, and will allow you to have properly-shadowed 2048x2048 res terrain at any squaresize. Yeah.

I'm sure there was a reason this code was done the way it originally was.. but here goes (line 679 in terrCell.cpp):

F32 screenError;
   if ( state->isShadowPass() )
      screenError = F32_MAX;
   else
      screenError = terrain->getScreenError();

This looked pretty suspicious. I mean, how often do you see F32_MAX?

Now, I'm a little confused. Are we trying to force the lowest LoD for terrain render? That's not going to work out well. Realistically, we want to use the player's current viewpoint to get the LoD, just like the regular geometry does. I mean, if you're standing in spot X, you want the shadows from the terrain that appear there be be rendered from what the terrain looks like at spot X, right? So they match up?

F32 screenError;
   if ( state->isShadowPass() )
      screenError = terrain->getScreenError();
   else
      screenError = terrain->getScreenError();

You could remove the entire if() bit, but I was trying to preserve the structure of the original code. The truth is, I went ahead and changed F32_MAX to 0.. I thought of two things here: 1) this might have been what was intended by F32_MAX, 2) using terrain->getScreenError() still allowed this effect to appear in extreme cases when low-LoD was used on terrain features near and behind the player.

Anyway, I'm sure people will be along shortly to tell me what this breaks and why it was like that, but until then, enjoy this 16kmX16km desert environment with perfect shadowing.


The real issue here isn't specifically that it was using a high bias toward low terrain LoD, it's that shadow rendering at the lowest terrain LoD fails significantly.

There's actually one other shadow glitch, but it's related to extremely large terrain structures casting shadows across cell boundaries and it's not really worth worrying about right now. Some dynamic control of the cell size would solve this issue (ie, at a terrain of size 32x32km (4096), we don't need 64 block cells). Just be aware of where your cells are when building really complicated and large mountains.

#7 That is actually a really good fix!
Nice find, I was playing with code right near there last night trying to find a way to "LOD TO HELL" my faraway terrain blocks. Never thought about shadows in that area...
Way to stick to it!

About your extra large cell size terrains, once terrain gets that big, one could always use a smaller hightmap to build the mountains and details.


EDIT 60minuts later: I also find screenError = 0; to have best results in place of F32_MAX;.

#8 Yeah, I can't be certain, but it would make sense that when writing this code, someone mixed up 0 with max, forgetting which one would force which thing. I'm hoping someone will fill us in on Monday on what's actually going on. I'm using 0 at the moment, and put the cell size back to 64.

#9 It will be Tuesday an Monday is a holiday in the States. I have been using screenError = 0; all after you had posted the 'fix' and dont seem to have any aftereffects.

It would be neat to know the what/how is happening.

#10 Setting the screenError to F32_MAX seems like it would force the shadows to be rendered from low LoD terrain geometry. If you set your screenError very high on your terrain, that decreases the distance within which it renders full detail terrain. For some reason, the shadow rendering for low detail terrain is making up those random blobs. :P

#11 @Henry - Yea... that was the bug. The code wanted to force the highest resolution terrain cells... not the lowest... and F32_MAX gave the opposite result.

Great catch!

Now one thing i wanted to experiment with was using the real screen error during shadow rendering, but was unsure what the results would be. Did you try running it that way?

#12 Ok... this is fixed for beta 3.

And from testing so far it looks like using LOD in the terrain shadows works fairly well with very few artifacts. The change is a bit more complex but here it is...

Replace TerrCell::cullCells() in terrain/terrCell.cpp with this one...

void TerrCell::cullCells(  TerrainBlock *terrain,
                           const Frustum *culler, 
                           const SceneState *state,
                           const Point3F &objLodPos, // CHANGE
                           Vector<TerrCell*> *outCells  )
{
   // If we have a VB and no children then just add 
   // ourselves to the results and return.
   if ( mVertexBuffer.isValid() && !mChildren[0]  )               
   {
      outCells->push_back( this );
      return;
   }

   const F32 screenError = terrain->getScreenError(); // CHANGE

   for ( U32 i = 0; i < 4; i++ )
   {
      TerrCell *cell = mChildren[i];

      // Test if visible.
      if ( !culler->intersects( cell->getBounds() ) )    
         continue;
      
      // Lod based on screen error...
      // If far enough, just add this child cells vb ( skipping its children ).
      F32 dist = cell->getBounds().getDistanceToPoint( objLodPos ); // CHANGE
      F32 errorMeters = ( cell->mSize / smMinCellSize ) * terrain->getSquareSize();
      U32 errorPixels = mCeil( state->projectRadius( dist, errorMeters ) );

      if ( errorPixels < screenError )
      {
         if ( cell->mVertexBuffer.isValid() )
            outCells->push_back( cell );       
      }
      else      
         cell->cullCells( terrain, culler, state, objLodPos, outCells ); // CHANGE
   }
}

... and in TerrainBlock::_renderBlock() in terrain/terrRender.cpp:

void TerrainBlock::_renderBlock( SceneState *state )
{
   PROFILE_SCOPE( TerrainBlock_RenderBlock );

   MatrixF worldViewXfm = GFX->getWorldMatrix();
   worldViewXfm.mul( getRenderTransform() );

   MatrixF worldViewProjXfm = GFX->getProjectionMatrix();
   worldViewProjXfm.mul( worldViewXfm );

   const MatrixF &objectXfm = getRenderWorldTransform();

   Point3F objCamPos = state->getDiffuseCameraPosition(); // NEW
   objectXfm.mulP( objCamPos ); // NEW

   if ( !mDefaultMatInst )
      mDefaultMatInst = MATMGR->createMatInstance( "TerrainShadowMaterial", getGFXVertexFormat<TerrVertex>() );

... and further below that near line 320...

if ( state->isDiffusePass() )
      mCell->clearRenderedFlag();

   mCell->cullCells( this,
                     &frustum, 
                     state,
                     objCamPos, // ADDED
                     &renderCells );

   if ( state->isDiffusePass() )
   {

If any of you could test this with your larger terrains it would be much appriciated.

#13 **edit
+update the void TerrCell::cullCells(...) function in terrCell.h

Seems to work fine on a 512x512x8.
Also looks good on a 2048x2048x8.

That seems to do it.

#14 Yey... well done Henry & Tom

I've not implemented the final fix yet, but trust you guys enough
to know it probably does as it says on the tin.

Well done everyone.

#15 @Henry - Thanks... yea it seems to be working here for a few different terrain sizes as well.