How To: Player Shadows on Poly-soup Meshes

So I am finally able to cast player shadows on meshes with poly-soup collision. This method, however, is a hack, and I make no guarantees about its correctness. That said, here is how I did it:

I started by integrating the Poly Soup resource.

I soon realized that the major problem with the resource is that it may not return the triangle vertices in the correct order. It seems that not all are front-facing polygons. To fix that, I removed all of the code that culls back facing polys. The only other problem I encountered had to do with building the depth partition. The clipping process was cutting out polys... so I skipped it. I just forced all shadow polys into the depth partition.

Again, this is a hack. I am sure there is a better way to solve this problem, but this is a rough start for anybody interested.

I then made these changes to the code:

In collision/clippedPolyList.cc
comment out these lines (144 - 151)

/*
   // Anything facing away from the mNormal is rejected
   if (mDot(poly.plane,mNormal) > 0) {
      mIndexList.setSize(poly.vertexStart);
      mPolyList.decrement();
      return;
   }
*/

in collision/depthSortList.cc
comment out these lines (105-112)

/*
   if (mPolyList.last().plane.y >= -MIN_Y_DOT)
   {
      mIndexList.setSize(mPolyList.last().vertexStart);
      mPolyList.decrement();
      return;
   }
*/

and finally, in shadow.cc
change line 21 to

U32 Shadow::smShadowMask = TerrainObjectType | InteriorObjectType | StaticShapeObjectType | StaticObjectType;

Thanks Max.

change line 520 from

ClippedPolyList::allowClipping  = (dynamic_cast<InteriorInstance *>(obj) != NULL);

to

ClippedPolyList::allowClipping = true;

and change line 480 from

smDepthSortList.depthPartition(gShadowPoly,4,mPartition,mPartitionVerts);

to

//force all polys into the shadow partition
	DepthSortList::Poly* polyPtr = 0;
	for( S32 i = 0; i < smDepthSortList.mPolyList.size(); i++ )
	{
		polyPtr = &smDepthSortList.mPolyList[i];
		mPartition.push_back( *polyPtr );
		for( S32 j = 0; j < polyPtr->vertexCount; j++ )
		{
			U32 index = smDepthSortList.mIndexList[ polyPtr->vertexStart + j ];
			mPartitionVerts.push_back( smDepthSortList.mVertexList[index].point );
		}
	}

And that should do it.