TGEA 1.8.0 BUG Atlas Terrain failed wrtiting bitmap

#1 Confirmed. I get the same issue. And unfortunately, I don't have version 1.7 to fall back on. Is there another workaround available, or am I unable to move on with my project until there's a patch? (or I guess I could get into the source code. But I don't really feel ready for that right now. Anyone willing to take a look at this yet?)

#2 It might help to post your full script code.
$Opacity and $OpacityAtlas may not be defined properly.

#3 It works in TGEA 1.7. Has something changed in how that is called? Looking at the actual code, it appears they made a few changes, but more in line with the new way they are programming in 1.8. But the inputs appear the same, if I recall correctly.

#4 This is the entire script from L3DTio_Atlas2_export_script.cs, which is created automatically by L3DT (the exporter plugin, anyway). Dunno how much it will help, since apparently it works with v1.7 of TGEA (haven't seen that personally, but I trust that Randy knows what he is talking about). Given that, it must be something in the TGEA code rather than the script, I would think.

//This script was generated by the L3DTio_Atlas2 plugin for L3DT
//For more info, please see <http://www.bundysoft.com/wiki/doku.php?id=plugins:fileio:L3DTio_Atlas2>
enableWinConsole(true);
setLogMode(6);
cls();
echo("Starting Atlas Build!\n\n");

// make geometry CHU and atlas
atlasOldGenerateChunkFileFromRaw16("Generated Terrains/deleteme/HF.raw", 512, 1.000000, 0.001410, "Generated Terrains/deleteme/HF.chu", 0.250000, 3);
importOldAtlasCHU("Generated Terrains/deleteme/HF.chu", "Generated Terrains/deleteme/geom.atlas");

// make texture map atlas
atlasGenerateTextureTOCFromTiles(4, "Generated Terrains/deleteme/TX/TX_x%dy%d", "Generated Terrains/deleteme/tex.atlas", 0);

// make unique textured atlas
atlasGenerateUniqueTerrain("Generated Terrains/MyTerrain1.atlas", "Generated Terrains/deleteme/geom.atlas", "Generated Terrains/deleteme/tex.atlas");

echo("Build complete!");
quit();

this is the console.log file, which is just a run of the above script. The last line is where it got to before it crashed, it would appear...

//-------------------------- 1/1/2009 -- 10:23:50 -----
Starting Atlas Build!


Generating chunked geometry (depth=3, baseMaxError=0.250000)
   o Calculating activation levels...
      - done.
   o Propagating activation levels...
      - done.
   o Writing file header...
      - done.
   o Generating empty TOC...
      - done.
   o Generating meshes...
AtlasOldActivationHeightfield::generateNodeData - Max exceeded! May have paging issues!
AtlasOldActivationHeightfield::generateNodeData - Min exceeded! May have paging issues!
AtlasOldActivationHeightfield::generateNodeData - Max exceeded! May have paging issues!
      - done.
=== Chunk Statistics ===\
\===== Level    Count    Avg. Size
            0          16      3701.750000
            1           4      14364.250000
            2           1      33817.000000
========================================
                chunks:         21
           input verts:     262144
          output verts:      18006

       avg verts/chunk:        857
NOTE: verts/chunk is low; for higher poly throughput consider setting the tree depth to 2 and reprocessing.
          output bytes:        805
      bytes/input vert:       0.00
     bytes/output vert:       0.04
        real triangles:     150502
AtlasDeferredFile::ensureStreamWritable - switching file into writable mode... This may cause a crash if you have other copies of Torque accessing it!
importOldAtlasCHU - created new Atlas file 'Generated Terrains/deleteme/geom.atlas'
importOldAtlasCHU - Atlas file initialized, converting...
importOldAtlasCHU - Headers read, remapping chunks...
importOldAtlasCHU - Importing geometry chunks...
importOldAtlasCHU -    level 2 chunks...
importOldAtlasCHU -    level 1 chunks...
importOldAtlasCHU -    level 0 chunks...
AtlasFile::waitForPendingWrites - Waiting for pending output to finish.
AtlasFile::waitForPendingWrites - Done!
importOldAtlasCHU - import OK!
AtlasFile::waitForPendingWrites - Waiting for pending output to finish.
AtlasFile::waitForPendingWrites - Done!
atlasGenerateTextureTOCFromTiles - Initializing atlas file 'Generated Terrains/deleteme/tex.atlas'...
AtlasDeferredFile::ensureStreamWritable - switching file into writable mode... This may cause a crash if you have other copies of Torque accessing it!
atlasGenerateTextureTOCFromTiles - Atlas started, processing tiles for 3 deep tree...
atlasGenerateTextureTOCFromTiles - Storing tiles in JPEG format...
atlasGenerateTextureTOCFromTiles - processing row 0...

#5 Thanks for posting Ron. I was going to run mine later and post (should say because I have made code changes since this, so need to restore). But I am using Grome, however the commands are identical. And from the console commands, stopping in the exact same location.


I would offer to convert yours for you in 1.7, but don't know a good way to transfer those files.


Randy

#6 No prob, Randy. I haven't settled on a terrain yet, so there's no real reason to send you my files anyway. I'm still goofing around with L3DT :)

I've isolated the code block where the crash occurs, it is in AtlasImportTiles.cpp, somewhere in here:

// Now, load the base chunks in - ie, the ones we have exact tiles for.
   char buff[512];
   for(S32 x=0; x<leafSize; x++)
   {
      Con::printf("atlasGenerateTextureTOCFromTiles - processing row %d...", x);

      for(S32 y=0; y<leafSize; y++)
      {
         // Load the image...
         S32 imgX, imgY;
         imgX = y;
         imgY = ((leafSize-1) - x);

         dSprintf(buff, 512, tileMask, imgX, imgY);
         GBitmap *bmp = GBitmap::load(buff);

         if(!bmp)
         {
            Con::errorf("atlasGenerateTextureTOCFromTiles - unable to open tile '%s'! Aborting!", buff);
            return;
         }

         // Create a chunk to store this data in.
         AtlasTexChunk *atc = new AtlasTexChunk;
         atc->mFormat = (AtlasTexChunk::TexFormat)outFormat;
         atc->bitmap = new GBitmap[ 1 ];
	      atc->bitmap[ 0 ] = *bmp;
	      SAFE_FREE( bmp );
	      atc->layerCount = 1;
         
         // Get the stub.
         AtlasResourceTexStub *tStub = arttoc->getStub(treeDepth-1, Point2I(x,y));

         // And store the chunk.
         arttoc->instateNewChunk(tStub, atc, true);

         // Purge it once we're done so we don't use gigs of ram!
         tStub->purge();
      }
   }

   arttoc->generate(RectI(0, 0, leafSize, leafSize));

   af.waitForPendingWrites();

   Con::printf("atlasGenerateTextureTOCFromTiles - Done.");

I don't have any way to step through this code yet (I only have Builder on my machine right now). I may download MS Express tomorrow and see if I can get a debug build put together and perhaps I can isolate the problem down a bit more. Do you have the equivelent code block from 1.7 Randy? What has changed?

#7 I will look and see..

#8 Only differences are below...

1.7....

#include "console/console.h"
#include "gfx/gBitmap.h"

#include "atlas/core/atlasFile.h"
#include "atlas/resource/atlasResourceTexTOC.h"

1.8....


#include "core/strings/stringFunctions.h"

#include "atlas/resource/atlasResourceTexTOC.h"

I added the following and recomiled, but still same error.

#include "console/console.h"
#include "gfx/bitmap/gBitmap.h"
#include "atlas/core/atlasFile.h"

However, I do get the following error when I run debug, this is before and after the above changes.

Fatal: (f:\torque\tgea_1.8\trunk\engine\source\gfx\bitmap\loaders\bitmapjpeg.cpp @ 164) GBitmap::writeJPEG: ONLY RGB bitmap writing supported at this time.

#9 There seems to be a comment in the code about how gBitmap::load works differently now.

/// Load the given bitmap file.
///
/// Important: Don't do something like this
///
/// @code
///    GBitmap* bitmap = GBitmap::load( filename ); // DON'T DO!!!
/// @endcode
///
/// Resources are reference-counted and the smart pointer conversion will
/// release the bitmap and thus render the resulting bitmap pointer invalid!

Resource<GBitmap> GBitmap::load(const Torque::Path &path)
{

After running a debug of an atlas build it appears exactly what the comment is warning about is happening.
The loaded bitmap pointer becomes invalid.

Not sure what the proper way to use this now is.

GBitmap *bmp = GBitmap::load(buff);

#10 That is actually what is happening. Good find though, I skimmed that the other night, but was way to tired at that point.

#11 Indeed, nice catch there Bill. I did a quick search for the call in the source files, and there is a lot of this going on:

Resource bitmap = GBitmap::load( correctPath );

That one is from gfxTextureManager.cpp

However, in gFont.cpp there is also this line:

GBitmap *strip = GBitmap::load(fileName);

So if it's a bug to call GBitmaps load function in that manner, it appears it's in at least two places.

I'm going to change the line in AtlasImportTiles.cpp to make it a resource, and see what happens.

#12 I changed the line to

Resource< GBitmap > bmp = GBitmap::load(buff);

Got the same crash :(

What is the command line argument that will allow me to run AtlasDemo with the L3DTio_Atlas2_export_script.cs script instead of main.cs? I need to run this in the compiler's IDE in order to step through the code to get to the offending line.

#13 Not sure. I just rename my gromeout.cs to main.cs and back up my main.cs to main_orig.cs.. Then just rename em back when I need to.. I have one project setup specifically for exporting Atlas, so I never use the actual original main.cs in that project anyway.

OH.. Yep, I had tried that as well and it wouldn't compile..:)

#14 I moved L3DTio_Atlas2_export_script.cs to the game\scriptsAndAssets\server folder and added

exec("./server/L3DTio_Atlas2_export_script.cs");

to the start of function onStart() in the main.cs file in scriptsAndAssets folder.

You can just comment it out later when your done with it.

#15 Hehe, many ways to do it. It turns out if you run your program with the first command line argument as a script file name, it will run that script rather than main.cs. So I just plugged L3DTio_Atlas2_export_script.cs into VC as an argument and it worked. Pretty cool how they set that up.

Well guys, I can't get it to work. I think you are right, Bill, that's the issue, but I don't know how to fix it.

This was my last attempt. It runs all the way to the end of the first inner loop, then crashes the second time through the loop right at the end. The compiler is stopped at GBitmap::deleteImage(), so something didn't reset right the first time through. Somebody take a look at it and see if you see something I'm missing.

// Now, load the base chunks in - ie, the ones we have exact tiles for.
   char buff[512];
   Resource< GBitmap > bmp;
   for(S32 x=0; x<leafSize; x++)
   {
      Con::printf("atlasGenerateTextureTOCFromTiles - processing row %d...", x);

      for(S32 y=0; y<leafSize; y++)
      {
         // Load the image...
         S32 imgX, imgY;
         imgX = y;
         imgY = ((leafSize-1) - x);

         dSprintf(buff, 512, tileMask, imgX, imgY);
		 bmp = GBitmap::load(buff);

         if(!bmp)
         {
            Con::errorf("atlasGenerateTextureTOCFromTiles - unable to open tile '%s'! Aborting!", buff);
            return;
         }

         // Create a chunk to store this data in.
         AtlasTexChunk *atc = new AtlasTexChunk;
         atc->mFormat = (AtlasTexChunk::TexFormat)outFormat;
         atc->bitmap = new GBitmap[ 1 ];
	     atc->bitmap[ 0 ] = *bmp;
	     atc->layerCount = 1;
         
         // Get the stub.
         AtlasResourceTexStub *tStub = arttoc->getStub(treeDepth-1, Point2I(x,y));

         // And store the chunk.
         arttoc->instateNewChunk(tStub, atc, true);

         // Purge it once we're done so we don't use gigs of ram!
         tStub->purge();
      }
   }

#16 We might be back to pre-1.7.1??

http://www.garagegames.com/blogs/985/14879

Quote:GBitmap::findBmpResource() and GBitmap::load() now properly attempt to load the file with the given name and then will try to load with all of the other valid texture extensions. Before it was simply appending the different extensions to the full file name so you would get things like "demoWindow.jpg.png". This code was made a lot cleaner by using the stripExtension() and addExtension helper functions mentioned above.

There is no stripextension() in gBitmap.ccp (which is now 991 lines as compared to 1310).

Bill, the line you mention above, is on 1.7.1. That whole file is almost completely identical (which does not mean much I know).

#17 Hey guys. Check this out.....:)

tdn.garagegames.com/wiki/TorqueGameEngineAdvanced/1_8_Beta/Porting/Gotchas

Quote:Resource Loading Change

The new Resource Manager introduces a "Gotcha", which directly affects the loading of texture files, such as bitmaps and DDS. Previously, the GBitmap and DDSFile classes would return a class pointer. For example:

GBitmap* bitmap = GBitmap::load( path );


This is no longer the case. The new Resource Manger actually returns a resource handle. The GFXTextureManager class has a perfect example. An extremely important class function GFXTextureManager::createTexture(...). Within this function, we can see the Resource Manager in action:

// MP: Load a DDS file
Resource dds = DDSFile::load( path );

// MP: Load a bitmap file
Resource bitmap = GBitmap::load( path );


As you can see, instead of returning a class pointer (GBitmap* or DDSFile*), the ::load function now returns a handle to the resource, via a templated class that acts as a wrapper (Resource).

@Ron, looks like i missed you last post. May try that code tomorrow. Moving on to other things tonight.

#18 Okay, so if we are now getting a resource handle, I expect this line is now incorrect

atc->bitmap[ 0 ] = *bmp;

atc->bitmap is expecting a pointer to a GBitmap class, and instead what we are giving it is a resource handle, so that can't be right. How do we fix this?

#19 Sounds good Randy.

I think I've gotten a bit off track. It seems to me that the entire point of the original line, which was

GBitmap *bmp = GBitmap::load(buff);

was to get a pointer to a GBitmap class. The only difference is now is that we are getting a Resource Handle. There obviously must be a way to get back to the resource handle's class pointer (which in our case would be a GBitmap). If we could do that, we're back to the original functionality, and everything else can be left as is.

What we need is to change the line

GBitmap *bmp = GBitmap::load(buff);

into two lines

Resource = GBitmap::load(buff);
GBitmap *bmp = bitmap->GetClassPointer();

Or whatever the correct function happens to be. Then we are back to the functionality we had in version 1.7

At least, I think that's what we need.

#20 I think you are right, the key may be in here, that is where they have something similar setup.

GFXTextureManager::createTexture