Finding Memory Leaks Made Easy

Memory leaks are a pain. Every developer hates them, and they can be very difficult to track down. Torque has some tools that make it much easier to find memory leaks quickly and easily that you may not know about.

When you compile Torque, (or just platformMemory.cc) with DEBUG_GUARD defined, it stores an extra chunk of information in the allocated memory header such as the file name and line number of the memory allocation. This is very useful when combined with the ability to set flags on allocated memory. The method of memory leak detection that I am presenting is based off of two (script) function calls:
FlagCurrentAllocs and
DumpUnflaggedAllocs
When FlagCurrentAllocs is called, the method steps through every allocated memory header, and sets a flag that basically just says, "Hey this header has been flagged." When DumpUnflaggedAllocs is called, it steps through all the headers, and any header that does not have the FlaggityFlag set will be dumped to a file in the following format:
FileName line size-in-bytes alloc-number

"Great," you say, "How does that help me?" Well believe it or not, this is really, super-useful and powerful. The best way to demonstrate this, however, is with an example! I added this function to consoleFunctions.cc

ConsoleFunction( leakMemory, void, 1, 1, "" )
{
   char *foo = new char[256];
   Con::printf( "--Leaked 256 * sizeof( char ) bytes of memory!" );
}

Make sure Torque is compiled with DEBUG_GUARD defined in platformMemory.cc or globally. (For the purposes of this tutorial, it is ok to just uncomment that line in platformMemory.cc, but if you are looking for one in your game, it should be defined in the project file. Now do the following:
1. Start up Torque.
2. Pop down the console
3. Type: FlagCurrentAllocs();
4. Type: DumpUnflaggedAllocs("./dump0.txt");
5. Type: leakMemory();
6. Type: DumpUnflaggedAllocs("./dump1.txt");

The contents of dump0.txt should look like this:

Quote:
i:\source\torque\engine\console\console.cc 777 56 4363
i:\source\torque\engine\console\compiler.cc 610 20 4366
i:\source\torque\engine\console\compiler.cc 510 28 4367
i:\source\torque\engine\console\consoleinternal.cc 280 60 4369
i:\source\torque\engine\console\consoleinternal.cc 249 32 4370
i:\source\torque\engine\console\consoleinternal.cc 395 48 4371
i:\source\torque\engine\console\console.cc 777 56 4372
i:\source\torque\engine\console\compiler.cc 610 32 4375
i:\source\torque\engine\console\compiler.cc 510 40 4376

i:\source\torque\engine\console\consoleinternal.cc 442 32 4368

Anything that comes from consoleInternal.cc, console.cc, or compiler.cc, is NOT a memory leak (most likely, unless you broke something in there) because it is mostly reallocations and allocations from script memory.

The contents of dump1.txt should look like this:

Quote:
i:\source\torque\engine\console\consolefunctions.cc 27 256 4362
i:\source\torque\engine\console\console.cc 777 56 4363
i:\source\torque\engine\console\compiler.cc 610 20 4366
i:\source\torque\engine\console\compiler.cc 510 28 4367
i:\source\torque\engine\console\consoleinternal.cc 280 60 4369
i:\source\torque\engine\console\consoleinternal.cc 249 32 4370
i:\source\torque\engine\console\consoleinternal.cc 395 48 4371
i:\source\torque\engine\console\console.cc 777 56 4372
i:\source\torque\engine\console\compiler.cc 610 32 4375
i:\source\torque\engine\console\compiler.cc 510 40 4376
i:\source\torque\engine\console\consoleinternal.cc 442 32 4368

Hey, there's our memory leak! The new call was on line 27, and 256 bytes were allocated, it was allocation #4362. See how this could be useful? But wait there's MORE!

What is that allocation number, and why is it useful? Well, I'll tell you! Do the following:
1. Start up Torque with the following command line argument: -jSave foo.jrn
2. Pop down the console
3. Type: FlagCurrentAllocs();
4. Type: leakMemory();
5. Type: DumpUnflaggedAllocs("./jrn_dmp.txt");

Look at the memory leak line, note the allocation number, and change the following line in platformMemory.cc

U32  gBreakAlloc = 0xFFFFFFFF;

to

U32  gBreakAlloc = (your allocation number);

(Note this is the hackey way to do it, there is a function called setBreakAlloc to do this)

Now run Torque with the following command line argument in the debugger: -jPlay foo.jrn

If you've never seen a journal running...well now you've seen one. Cool huh. Now when it executes the 'leakMemory();' line, it will debug break. This will let you break on a certain allocation. But wait there's more!

One issue I ran into on Marble Blast for the Xbox was a GPU memory leak, so all I'd get was that there was an un-flagged TextureHandle. This happened sometime during a mission, so what I did was:
1. Start Marble Blast, wait for a demo to start playing
2. Exit mission
3. Flag open allocs
4. Wait for another demo to start playing
5. Exit mission
6. Dump unflagged allocs

Now not every item in gTexManager was a memory leak because when the mission loaded, the bitmaps used for the main menu were unloaded and reloaded, so I had to figure out which ones were actually leaks, and which were just loaded when the main menu came back up, so what I needed to know was the texture name...and this is how I did it:

if( pah->fileName && dStrstr( pah->fileName, "gTexManager.cc" ) > 0 )
{
   U8 *foo = (U8 *)pah;
   foo += sizeof(Header);
   TextureObject *obj = (TextureObject *)foo;

   dSprintf( buffer, sizeof( buffer ), "Possible Texture Leak: %s\n", obj->texFileName );

   OutputDebugString( buffer );
}

So that's about it for memory leak detection. You will get an idea after a while of what actually is a leak, and what is a reallocation or a console allocation. I also recomend pasting the information into a spreadsheet so you can sort by file name etc. I hope this helps some of you. In case you have a version of Torque without the code I am talking about, all you have to do is paste the following code into platformMemory.cc:

Add the bold line to the MemConstants enum:

enum MemConstants {
   Allocated            = BIT(0),
   Array                = BIT(1),
   [b]FlaggityFlag         = BIT(2),[/b]
   AllocatedGuard       = 0xCEDEFEDE,
   FreeGuard            = 0x5555FFFF,
   MaxAllocationAmount  = 0xFFFFFFFF,
   TreeNodeAllocCount   = 2048,
};

Add the following code right before the initLog() function:

void flagCurrentAllocs()
{
   PageRecord* walk;
   for (walk = gPageList; walk; walk = walk->prevPage) {
      for(Header *probe = walk->headerList; probe; probe = probe->next)
         if (probe->flags & Allocated) {
            AllocatedHeader* pah = (AllocatedHeader*)probe;
            pah->flags |= FlaggityFlag;
         }
   }
}

#ifdef DEBUG_GUARD
ConsoleFunction(FlagCurrentAllocs, void, 1, 1, "FlagCurrentAllocs();")
{
   argc; argv;
   flagCurrentAllocs();
}

ConsoleFunction(DumpUnflaggedAllocs, void, 2, 2, "DumpUnflaggedAllocs(filename);")
{
   argc;
   FileStream fws;
   fws.open(argv[1], FileStream::Write);
   char buffer[1024];

   PageRecord* walk;
   for (walk = gPageList; walk; walk = walk->prevPage) {
      for(Header *probe = walk->headerList; probe; probe = probe->next)
      {
         if (probe->flags & Allocated) {
            AllocatedHeader* pah = (AllocatedHeader*)probe;
            if (!(pah->flags & FlaggityFlag))
            {
               // If you want to extract further information from an unflagged
               // memory allocation, do the following:
               // U8 *foo = (U8 *)pah;
               // foo += sizeof(Header);
               // FooObject *obj = (FooObject *)foo;
               dSprintf(buffer, 1023, "%s\t%d\t%d\t%d\r\n",
                  pah->fileName != NULL ? pah->fileName : "Undetermined",
                  pah->line, pah->realSize, pah->allocNum);
               fws.write(dStrlen(buffer), buffer);
            }
         }
      }
   }

   fws.close();
}

// --- Put the rest of the #ifdef DEBUG_GUARD code here