Unit Tests

#21 Mm, good point - we could replace the leaking parts of the code with fixed-size buffers. I'm not sure what's good practise. I suspect, though, that the console itself needs the buffer to operate as an in-game UI. If you turn the buffer off, then you see nothing in the console. So I guess a fixed-size buffer would mean a fixed-length history in the game UI, which isn't all that terrible.

Oh, I just realised that's what you were suggesting. Yeah, that'd be a good option, though I just realised that you're still allocating new strings whenever you print to the console anyway. Unless you make each line a fixed length and pre-allocate that as well. I'd much rather just turn the buffer off for unit test runs. It's easy enough to read the log in the VS output window.

The profiler, unfortunately, is probably unavoidable, because it needs to dynamically build up the profile graph as it operates. The problem I'm running into right now is that for some reason, it gets turned on during engine startup, and I can't immediately turn it off. If the profiler is off then we're fine, I think, but it won't turn off on-demand.

Okay, so the profiler stays off and the tests run fine if you comment this out in torqueConfig.h:

#define TORQUE_ENABLE_PROFILE_PATH

which disables this block:

void flagCurrentAllocs( EFlag flag )
{
#ifdef TORQUE_ENABLE_PROFILE_PATH
   if (gProfiler && !gProfiler->isEnabled())
   {
      gProfiler->enable(true);
      // warm it up
      //gProfiler->dumpToConsole();
   }
#endif

...which is called from StandardMainLoop::init. So this is basically doing memory profiling, AFAICT. Hmm.

EDIT: I should be clear that there are only memory leaks when you consider the scope of a single test. For example, if a test runs and prints output to the console, it has created a 'leak' because memory was allocated for that console log and was not freed before the test ended. It's not a leak in the application, because that memory will be freed at some point later on, when the console decides it's done. In the same way, memory allocated by the profiler is freed when the profiler dumps or whenever.

EDIT: Also, I've sidestepped the problem for now by doing this:

$Con::LogBufferEnabled = false;
$Testing::CheckMemoryLeaks = true;
profilerEnable(false);
function runTests() {
   runAllUnitTests();
   quit();
}
schedule(1000, 0, runTests);

I've just given the profiler 1 second to allow it to close down before running tests :P.

#22 Ok, so the memory profiler needs to be "console aware."

I think I agree with your assessment that it's easier by far to disable the console during the tests and read the VS output. Especially if the unit tests and memory profiling can be run automatically. Even more especially since this is only done in development and it's not for production distribution use.

The main thing to keep in our heads as we work is not to #ifdef stuff out for debug that comes on in release that bypasses our tests....

#23 Exactly, that's why I wanted a solution that didn't require changing the #defines. At this point, you can also disable memory leak checking, so the tests can be run without the profiler or console getting in the way. But if we ever wanted to unit test the profiler or console themselves, and check for memory leaks, we'd have to do something more radical. It'd even be possible to create a completely separate compile that didn't do all the regular main loop stuff, which would be just for testing those core classes in isolation. That may even be desirable. I don't know.