TGEA 1.7.0 Bug: atlasGenerateTextureTOCFromLargeJPEG crashes
by Henri Aalto · in Torque Game Engine Advanced · 04/07/2008 (2:20 pm) · 19 replies
I don't know what's wrong but I simply can't get the "atlasGenerateTextureTOCFromLargeJPEG" to work in any of the TGEA 1.7.0 releases (betas 1 & 2 nor the official release). It simply crashes the game with any of the supplied .exe files.
I rebuilt the project in debug mode and it fails in dbgdel.cpp with assert for "_BLOCK_TYPE_IS_INVALID(pHead->nBlockUsed)" in atlasResourceTOC.h at:
void purge()
{
// Blast the chunk.
if(mChunk)
{
delete mChunk;
mChunk = NULL; << == crashes here
// Getting hot!
mHeat += 0.5f;
}
mState = Unloaded;
}
If I change the order of the "delete" and "=null" lines, it works until it fails in gBitmap.h "getWidth" function":
inline U32 GBitmap::getWidth(const U32 in_mipLevel) const
{
AssertFatal(in_mipLevel < numMipLevels,
avar("GBitmap::getWidth: mip level out of range: (%d, %d)",
in_mipLevel, numMipLevels));
U32 retVal = width >> in_mipLevel;
return (retVal != 0) ? retVal : 1;
}
...arguing that the values were 0,0. It is likely called from atlasTexChunk.cpp (@ line 156):
// Prepare our own GBitmaps to write into.
U32 tileSize = children[0]->bitmap->getWidth();
It could be something in my compiler setup, but all the older versions of TGEA I've compiled work just fine, the problems infests only 1.7.0. I have november version of DirectX SDK and latest version of Windows SDK and generally I've set up the environment as stated in the TGEA 1.7.0 documents.
I rebuilt the project in debug mode and it fails in dbgdel.cpp with assert for "_BLOCK_TYPE_IS_INVALID(pHead->nBlockUsed)" in atlasResourceTOC.h at:
void purge()
{
// Blast the chunk.
if(mChunk)
{
delete mChunk;
mChunk = NULL; << == crashes here
// Getting hot!
mHeat += 0.5f;
}
mState = Unloaded;
}
If I change the order of the "delete" and "=null" lines, it works until it fails in gBitmap.h "getWidth" function":
inline U32 GBitmap::getWidth(const U32 in_mipLevel) const
{
AssertFatal(in_mipLevel < numMipLevels,
avar("GBitmap::getWidth: mip level out of range: (%d, %d)",
in_mipLevel, numMipLevels));
U32 retVal = width >> in_mipLevel;
return (retVal != 0) ? retVal : 1;
}
...arguing that the values were 0,0. It is likely called from atlasTexChunk.cpp (@ line 156):
// Prepare our own GBitmaps to write into.
U32 tileSize = children[0]->bitmap->getWidth();
It could be something in my compiler setup, but all the older versions of TGEA I've compiled work just fine, the problems infests only 1.7.0. I have november version of DirectX SDK and latest version of Windows SDK and generally I've set up the environment as stated in the TGEA 1.7.0 documents.
#2
I've experienced the same bug while trying to build Atlas terrain exported from Grome Pro.
With this change it works great now! Thanks!
04/07/2008 (3:34 pm)
Cheers Rene!I've experienced the same bug while trying to build Atlas terrain exported from Grome Pro.
With this change it works great now! Thanks!
#4
(This bug exists because at one time only a single "image plane" went with a tile... then it got changed to support more than one so you could have more than 4 opacity channels. Apparently not every code path that copied images got updated! I should have checked more thoroughly when I made that change. :)
04/07/2008 (9:29 pm)
Thanks for taking care of this Rene!(This bug exists because at one time only a single "image plane" went with a tile... then it got changed to support more than one so you could have more than 4 opacity channels. Apparently not every code path that copied images got updated! I should have checked more thoroughly when I made that change. :)
#5
04/07/2008 (11:10 pm)
@Rene: Thanks man, I'll check it out when I get back home. I really didn't have the time to start browsing trough all the generation code, nevertheless the Atlas code itself (a daunting task indeed!). I hope GG gives Atlas a bit more time because I think it has great potential (if it only was easier to use).
#6
Now that's a bug I *could* fix...
@Henri
Seems we're already two members here who think Atlas is the way to go. As for it being a daunting task to get into Atlas' code, I know too well what you mean. It's a complex beast. I am very thankful to the extremely helpful Ben Garney who has been constantly giving valuable advice to illuminate my way.
I can assure you, however, that GG *is* about to give Atlas more time. Expect some great updates!
And it *is* a great system.
By the way, have you seen Bill Vee's spherically projected stitched Atlas terrains. Awesome.
04/07/2008 (11:28 pm)
@BenNow that's a bug I *could* fix...
@Henri
Seems we're already two members here who think Atlas is the way to go. As for it being a daunting task to get into Atlas' code, I know too well what you mean. It's a complex beast. I am very thankful to the extremely helpful Ben Garney who has been constantly giving valuable advice to illuminate my way.
I can assure you, however, that GG *is* about to give Atlas more time. Expect some great updates!
And it *is* a great system.
By the way, have you seen Bill Vee's spherically projected stitched Atlas terrains. Awesome.
#7
This avoids the duplicate copying of the image data.
//Edit
In atlasImportTiles.cpp, the less efficient version (the way it is currently done in 1.7) has to be used as the bitmap construction is different there.
04/08/2008 (1:39 am)
By the way, my fix to the problem wasn't very efficient. Here's a better version:atc->bitmap = new GBitmap[ 1 ];
atc->layerCount = 1;
GBitmap *gb = &atc->bitmap[ 0 ];
new ( gb ) GBitmap( tileSize, tileSize );
for(S32 k=0; k<tileSize; k++)
dMemcpy(gb->getAddress(0, k), rows[k] + j*tileSize*3, 3*tileSize);This avoids the duplicate copying of the image data.
//Edit
In atlasImportTiles.cpp, the less efficient version (the way it is currently done in 1.7) has to be used as the bitmap construction is different there.
#8
It works!
But my next step was to generate the chunk file from a raw file with "atlasOldGenerateChunkFileFromRaw16" but it simply says "generateChunkFileFromRaw16 - failed to load 'tmp/wip16.raw'!". I've tried various directories (new and already existing) with various file names, then I traced the path & filename solving resDictionary.cpp @ line 82 (::find(..) -method) and it fails to find the files. Is there something specific about the placement or naming of the files or is the code simply broken.
04/08/2008 (11:49 am)
Doh!It works!
But my next step was to generate the chunk file from a raw file with "atlasOldGenerateChunkFileFromRaw16" but it simply says "generateChunkFileFromRaw16 - failed to load 'tmp/wip16.raw'!". I've tried various directories (new and already existing) with various file names, then I traced the path & filename solving resDictionary.cpp @ line 82 (::find(..) -method) and it fails to find the files. Is there something specific about the placement or naming of the files or is the code simply broken.
#9
What part of the lookup fails in the resource manager, finding the resource at all or finding the file on disk?
04/08/2008 (11:58 am)
As long as the the file is placed in the directory or subdirectory of the game executable, there actually shouldn't be a problem. The only time I had this happen was when I tried to run Torque from a path with a French accented character in it as there seem to be issues with Unicode conversion in Torque.What part of the lookup fails in the resource manager, finding the resource at all or finding the file on disk?
#10
P.S: my system is set up for finnish language and keyboard, don't know if that messes up something in the literals.
04/08/2008 (12:03 pm)
It seems to see the path and filename correctly but they don't seem to match anything in the hash-table. I'll give you an more accurate info once the rebuilding of the entire solution finishes.P.S: my system is set up for finnish language and keyboard, don't know if that messes up something in the literals.
#11
04/08/2008 (12:13 pm)
Check if the resource dictionary is empty. If it is, I bet all the cents I have left that it's a unicode conversion problem. Do you have non-english character in the path leading to your game directory? If so, try moving it out to e.g. "C:\Torque".
#12
Can't be that problem as you already ran the JPEG conversion from the same configuration, I assume.
Still the right thing to check the resource dictionary.
Stupid question: are you sure, your paths are really correct? I.e. that "tmp/wip16.raw" is directory subordinate to the "game/" folder where the executable resides?
04/08/2008 (12:16 pm)
Umpff... Me is an idiot!!Can't be that problem as you already ran the JPEG conversion from the same configuration, I assume.
Still the right thing to check the resource dictionary.
Stupid question: are you sure, your paths are really correct? I.e. that "tmp/wip16.raw" is directory subordinate to the "game/" folder where the executable resides?
#13
I created a new "tmp" folder under the game folder of atlas_demo and placed a "WIP16.raw" there. I just traced the situation with VS and ended resDirectory.cpp again:
ResourceObject* ResDictionary::find(StringTableEntry path, StringTableEntry name)
{
if(path)
{
char fullPath[1024];
Platform::makeFullPathName(path, fullPath, sizeof(fullPath));
path = StringTable->insert(fullPath);
}
for(ResourceObject *walk = hashTable[hash(path, name)]; walk; walk = walk->nextEntry)
if(walk->name == name && walk->path == path)
return walk;
return NULL;
}
The for-clause receives correctly solved name and path for the hash:
path = 0x01f71010 "C:/Torque/TGEA_1_7_0/GameExamples/AtlasDemo/game/tmp"
name = 0x01f71050 "wip16.raw"
... but the following if-branch is never executed as the walk->name seems to hold no valid data ==> the NULL is returned.
04/08/2008 (12:40 pm)
Yup.I created a new "tmp" folder under the game folder of atlas_demo and placed a "WIP16.raw" there. I just traced the situation with VS and ended resDirectory.cpp again:
ResourceObject* ResDictionary::find(StringTableEntry path, StringTableEntry name)
{
if(path)
{
char fullPath[1024];
Platform::makeFullPathName(path, fullPath, sizeof(fullPath));
path = StringTable->insert(fullPath);
}
for(ResourceObject *walk = hashTable[hash(path, name)]; walk; walk = walk->nextEntry)
if(walk->name == name && walk->path == path)
return walk;
return NULL;
}
The for-clause receives correctly solved name and path for the hash:
path = 0x01f71010 "C:/Torque/TGEA_1_7_0/GameExamples/AtlasDemo/game/tmp"
name = 0x01f71050 "wip16.raw"
... but the following if-branch is never executed as the walk->name seems to hold no valid data ==> the NULL is returned.
#14
The table is filled in one go at engine startup. For some reason, that filling may have failed to turn up proper directory contents.
It shouldn't matter, but from which directory are you running Torque? Are you running it from the command line in the game's own directory?
04/08/2008 (12:56 pm)
Hmm, have you checked the contents of the ResourceObjects it walks? Are there any at all? Is the hashtable empty?The table is filled in one go at engine startup. For some reason, that filling may have failed to turn up proper directory contents.
It shouldn't matter, but from which directory are you running Torque? Are you running it from the command line in the game's own directory?
#15
04/08/2008 (1:17 pm)
I tried running it from the windows explorer and the command shell at ""C:/Torque/TGEA_1_7_0/GameExamples/AtlasDemo/game" -folder. 
#16
04/08/2008 (2:08 pm)
You should try moving the tmp folder under game/scriptsAndAssets rather than directly under game.
#17
At this point I don't even ask for a deeper explanation, I'm just happy that it works. Thank you very much.
P.S: encountered another problem with the mission editor; when I edit the sun-object, altering its elevation and azimuth and then exit the editor with F11, TGEA asserts "bad" at missionEditor/editor.cpp @ 54, but lets me to continue normally.
04/08/2008 (2:50 pm)
@Matt: Ok -- that actually seemed to fix it! I also tried to run "atlasGenerateUniqueTerrain" against the game/tmp folder and it managed to find the geometry atlas but not the texture atlas even if they were both in the same directory (and the process to query them against the dictionary.hashtable is exactly the same).At this point I don't even ask for a deeper explanation, I'm just happy that it works. Thank you very much.
P.S: encountered another problem with the mission editor; when I edit the sun-object, altering its elevation and azimuth and then exit the editor with F11, TGEA asserts "bad" at missionEditor/editor.cpp @ 54, but lets me to continue normally.
#18
With TGE 1.5.0 I made it so that you could load files outside of the modPaths as long as you didn't enable the TORQUE_SHIPPING define in the C++ source code (which is something you would do before actually releasing your game). This probably was changed since then (it can introduce weird issues like having .DTS and .DIF files that are in your mission but not in your modPath) and I simply didn't get around to checking to make sure that the functionality still exists.
If you go into ResourceObject *ResManager::find (const char *fileName, bool noMissing) and remove the #ifdef TORQUE_TOOLS and the matching #endif then you should be able to build Atlas files with sources outside of the scriptsAndAssets, common, and tools folder.
04/08/2008 (4:40 pm)
The ResourceManager builds a list of all of the files in the "modPath" folders when the engine loads. By default the modPaths only include common, scriptsAndAssets, and tools so you will only be able to access files under those folders. This was done in Torque back in the Tribes 1 days to help protect against malicious scripts accessing files outside of the game directory (like in C:\Windows).With TGE 1.5.0 I made it so that you could load files outside of the modPaths as long as you didn't enable the TORQUE_SHIPPING define in the C++ source code (which is something you would do before actually releasing your game). This probably was changed since then (it can introduce weird issues like having .DTS and .DIF files that are in your mission but not in your modPath) and I simply didn't get around to checking to make sure that the functionality still exists.
If you go into ResourceObject *ResManager::find (const char *fileName, bool noMissing) and remove the #ifdef TORQUE_TOOLS and the matching #endif then you should be able to build Atlas files with sources outside of the scriptsAndAssets, common, and tools folder.
#19
04/08/2008 (4:45 pm)
Btw...please submit the other bug you found in its own thread using these guidelines. 
Associate Rene Damm
The initial error you get is because of a bug we already came across in the other texture TOC generation code. The problem is that atlasImportLargeImage.cpp incorrectly sets up the texture chunk. Here's the fix:
Replace "atc->bitmap = gb;" in atlasImportLargeImage.cpp@287 with:
-------------------
//EDIT
You can all give me a good beating, if you want. I already came across this in Beta 1 and simply marked it with a comment "//RD: same bug. not fixed". Argghh!!!