Game Development Community

TGEA 1.7.0 Beta 1 Bug - Crash in tiled texture import

by Rene Damm · in Torque Game Engine Advanced · 03/25/2008 (3:25 pm) · 6 replies

Importing of textures tiles for Atlas terrains may crash (console function atlasGenerateTextureTOCFromTiles). In release builds, you will get something like "Invalid stub" and in debug builds that include mem debugging, you'll see something like "Not an allocated block." The console log will show something like "... processing row 0..."

The reason is that atlasImportTiles.cpp improperly sets up its AtlasTexChunks. To fix this, replace the following in atlasImportTiles.cpp@89:

// Create a chunk to store this data in.
         AtlasTexChunk *atc = new AtlasTexChunk;
         atc->mFormat = (AtlasTexChunk::TexFormat)outFormat;
         atc->bitmap = *bmp;

with this code:

// Create a chunk to store this data in.
         AtlasTexChunk *atc = new AtlasTexChunk;
         atc->mFormat = (AtlasTexChunk::TexFormat)outFormat;
         atc->bitmap = new GBitmap[ 1 ];
	 atc->bitmap[ 0 ] = *bmp;
	 SAFE_FREE( bmp );
	 atc->layerCount = 1;

However, after applying this fix you will notice that it still crashes further down the line when trying to copy the texture TOC. This is because when copying the TOC, bitmap data is not duplicated and subsequently gets freed twice later on. So, to get done with this issue, replace the following in atlasTexChunk.cpp@251:

atc->bitmap[layer] = bitmap[layer];

with:

constructInPlace< GBitmap >( &atc->bitmap[layer], &bitmap[ layer ] );

You can do the same with the copying code for DDS files a few lines down in the same file, but DDSFile has no copy constructor. I am pretty sure, this is a bug as well and will cause the same problem when using DDS files.

And then, after spending hours (as I am completely new to Torque) tracking down and fixing these bugs and finally succeeding in generating an Atlas terrain, TGEA will then deadlock on loading the terrain. Looks like another round of bug hunting. Sigh...

-----------------------------
To finish this off: In order to solve the deadlock, reactivate the code that has been commented out in AtlasClipMapImageSource::isDataAvailable. What has been commented out there actually triggers load requests on unloaded data; without this, the loader code will wait forever for the data to get loaded.

So, the full code for the method in atlasClipMapImageSource.cpp@106 reads:

bool AtlasClipMapImageSource::isDataAvailable( const U32 mipLevel, const RectI inRegion )
{
   // We need to convert from a mip level to a level in our TOC, potentially scaling.
   U32 baseMips = getBinLog2(mTOC->getTextureChunkSize());
   U32 tocLevel = -1;
   if(mipLevel >= baseMips)
   {
      // In an inner or leaf tile.
      tocLevel = mipLevel - baseMips;
   }
   else
   {
      // It's in our base tile.
      tocLevel = 0;
   }
   
   // We don't scale beyond the depth of the TOC, so simply assert in that
   // case.
   AssertFatal(tocLevel < mTOC->getTreeDepth(), "AtlasClipMapImageSource::isDataAvailable - went beyond depth of tree.");

   // Check all the chunks of all the stubs in the appropriate region...
   RectI r;
   convertToTOCRect(tocLevel, inRegion, r);

   const S32 xStart = mClamp(r.point.x,               0, BIT(tocLevel));
   const S32 xEnd   = mClamp(r.point.x + r.extent.x,  0, BIT(tocLevel));

   const S32 yStart = mClamp(r.point.y,               0, BIT(tocLevel));
   const S32 yEnd   = mClamp(r.point.y + r.extent.y,  0, BIT(tocLevel));

   bool haveData = true;

   for(S32 x=xStart; x<xEnd; x++)
   {
      for(S32 y=yStart; y<yEnd; y++)
      {
         AtlasResourceTexStub *arts = mTOC->getResourceStub(mTOC->getStub(tocLevel, Point2I(x,y)));

         if(!arts->hasResource())
         {
            Con::printf("   - data unavailable for getStub(%d, (%d, %d)) %d", 
            tocLevel, x, y, mipLevel);

            mTOC->requestLoad(mTOC->getStub(tocLevel, Point2I(x,y)), AtlasTOC::NormalPriority,
                  F32(tocLevel) / F32(mTOC->getTreeDepth() + 1));

            haveData = false; 
         }

      }
   }

   return haveData;
}

#1
03/25/2008 (10:55 pm)
Awesome work Rene!

We honestly did not have a lot of time to work with Atlas. Just getting data into the system requires a fairly steep learning curve (something I hope to smooth out at some point...any ideas?).

It is my sincere hope to find a group of people who are passionate about Atlas who I can work closely with to further improve and develop it!
#2
03/25/2008 (11:42 pm)
Thanks, Matt.

It's sad to hear that Atlas is taking kind of a backseat. To me it seems a rather promising part of Torque. I am working towards having Atlas do large terrains with unique high-res texture+light+normal maps for me.

As for getting Atlas terrains into the engine, I don't think the current approach is bad. More documentation and more functionality of course would be great, but more in-engine editing for Atlas IMHO is non-sense. There are great tools for creating detailed high-res terrains out there and as long as there is some way of taking the various resulting maps and turning them into Atlas files, all is fine. If so inclined, one could whip up a simple GUI in Torque for driving the generation process.

PS: Why not work towards kicking legacy terrain out of Torque? Maintaining and supporting two terrain systems in a single engine seems a waste of resources.
#3
03/26/2008 (1:30 pm)
I think you just highlighted exactly why we are always going to need to two terrain systems =)

For some people, Atlas is a perfect fit and they are perfectly fine (and happy) using external tools to generate really, really big terrains with a lot of detail. This works great for some games and applications (simulations, military stuff, flight sims).

We thought that using and external tool and the turn-around lag that introduced was worth it for the detail and size of terrains you can get. After all, people don't seem to mind that same workflow for bringing art assets (like models and buildings) into the engine and even want us to further expand on that. For a lot of other people though, the ability to do real-time terrain editing is also crucial. Most of those people also just don't need huge terrains that can page off the hard drive. We've been getting that message loud and clear from developers with those needs so we decided to beef things up a little on that side for this round =)

Moving forward I would definitely like to work at making the two terrain systems share as much common code as possible (they are already sharing the same clipmapped texture renderer). It would be cool to see things like allowing MegaTerrains be paged off the hard drive but still real-time editable one day.

Right now my team hasn't had very much experience with Atlas and we have had problems laying hands on test data and people who properly understand it. We aren't pushing Atlas into the backseat, we just didn't have the time to get fully up to speed on it which is why it is geat that we have community members pitching in and helping! Please keep it up!
#4
03/27/2008 (10:23 am)
Hmm, reading through your post, I see your point and see that it indeed makes a lot of sense. Guess that in focusing on my own needs, I forgot that Torque isn't about suiting just one particular gametype/genre's needs.

So, looking forward to any improvements coming forth from your team's work and definitely hope to be able to contribute some improvements to Atlas of my own (next thing is custom lightmaps on unique terrains).
#5
03/31/2008 (10:44 am)
All of these bug fixes should be in Beta 2. Please retest if you get a chance. Thanks!
#6
04/01/2008 (8:21 am)
Yep, all working fine in Beta 2. Great!

By the way, I am convinced that the code I uncommented in the last part of my initial post actually should be commented out and is only necessary because of bugs in the Atlas I/O system (still hunting). However, as it only issues I/O requests, it probably doesn't hurt to leave it that way for now.