Torque X huge memory leak - Fixed
by Giuseppe De Francesco · in Torque X 2D · 03/18/2010 (11:54 am) · 298 replies
Well,
I'm not sure this is the right place to post this, maybe the blog? At any rate I'm posting here now.
Torque X has a huge memory leak, but seriously! I was working to a RPG game for a start so I didn't really notice, but 2 weeks ago I started YASS, a multiplayer side scrolling space shooter: a lot of particles, really a lot, and a lot of projectiles as well.
To let you understand here follows a short video of the game:
This is the memory situation after 3 minutes of gameplay (single player!!!) the game slowed down a lot:

So, I searched the forum and I realized that this issue is an old one and GG never got a grip on it (go figure why...) so I decided to fix the problems myself because I really need to publish YASS ;)
Long story short: I revised the whole engine and made a lot of fixes, some of them really of great importance, so here it is the situation playing my game (as you can see after 9 minutes the game still is 20% less than the original 3 minutes!):

There are still many stuff to fix, but at any rate in the current state I'm able to publish the game ;) I will make the SVN diff files available upon request (free of charge: pino AT dftgames DOT com).
Cheers,
Pino
I'm not sure this is the right place to post this, maybe the blog? At any rate I'm posting here now.
Torque X has a huge memory leak, but seriously! I was working to a RPG game for a start so I didn't really notice, but 2 weeks ago I started YASS, a multiplayer side scrolling space shooter: a lot of particles, really a lot, and a lot of projectiles as well.
To let you understand here follows a short video of the game:
This is the memory situation after 3 minutes of gameplay (single player!!!) the game slowed down a lot:

So, I searched the forum and I realized that this issue is an old one and GG never got a grip on it (go figure why...) so I decided to fix the problems myself because I really need to publish YASS ;)
Long story short: I revised the whole engine and made a lot of fixes, some of them really of great importance, so here it is the situation playing my game (as you can see after 9 minutes the game still is 20% less than the original 3 minutes!):

There are still many stuff to fix, but at any rate in the current state I'm able to publish the game ;) I will make the SVN diff files available upon request (free of charge: pino AT dftgames DOT com).
Cheers,
Pino
About the author
In the software eng. field since 1981, in charge of R&D during last 10 years. IEEE Senior Member (and volunteer).
#62
later I call
I have also tried
Still the object, SceneContainerBinReference found in TorqueCore/SceneGraph/SceneContainer.cs will stay in memory. The Memory profiler says it is an object "kept in memory by disposed objects". I am using the ANTS memory profiler to see this.
When running through the debugger the Dispose method in SceneContainerBinReference never gets called.
03/23/2010 (7:04 pm)
Scene containers are not releasing, I don't know if they ever did however in my game I call Game.Instance.SceneLoader.Load(@"datalevelsSpace_Map.txscene");
later I call
Game.Instance.SceneLoader.UnloadLastScene();
I have also tried
Game.Instance.SceneLoader.Unload(@"datalevelsSpace_Map.txscene"); and Game.Instance.SceneLoader.Unload(TorqueSceneData);
Still the object, SceneContainerBinReference found in TorqueCore/SceneGraph/SceneContainer.cs will stay in memory. The Memory profiler says it is an object "kept in memory by disposed objects". I am using the ANTS memory profiler to see this.
When running through the debugger the Dispose method in SceneContainerBinReference never gets called.
#63
I'm working exactly on that, it's the last issue left behind and it's very difficult to find out where on earth the live references are. The Bin reference system slowly grows whatever I do :( I'm tracing a schema using my preferred approach: paper and pencil :) diagram still growing, just like the bin reference :)
03/24/2010 (1:29 am)
Hi Henry, I'm working exactly on that, it's the last issue left behind and it's very difficult to find out where on earth the live references are. The Bin reference system slowly grows whatever I do :( I'm tracing a schema using my preferred approach: paper and pencil :) diagram still growing, just like the bin reference :)
#64
03/24/2010 (12:00 pm)
Ok, the Bin References look ok now, just waiting for Matthew feedback on loading, then I'll send this last diff files. This should wrap up the Core and TX2D major fixes, then I switch to check and fix the TX3D engine: I've seen already that has problems with the fixed Core... looks like TX3D was built using some Core bug as a feature :)
#65
03/24/2010 (12:05 pm)
I'll get you that feedback first thing when i get home. I have to work late tonight though so it wont be till 10 or so :).
#66
03/24/2010 (1:33 pm)
Cool, thanks!
#67
Hopefully these added lines wont cause any additional issues. :)
Thanks again! BTW: I noticed your from Ireland. I visited their recently and loved it. :)
03/24/2010 (7:04 pm)
Heya It works if I add additional null checks in the following blocks of codepublic int MaterialRegionIndex
{
get { return _materialRegionIndex; }
set
{
_materialRegionIndex = value;
if (_quad != null)
_quad.RefillVB = true;
}
}public override bool OnRegister()
{
if (!base.OnRegister())
return false;
if (_quad != null)
_quad.IgnoreTextureCoordsUpdate = true;
else
return false;
return true;
}Hopefully these added lines wont cause any additional issues. :)
Thanks again! BTW: I noticed your from Ireland. I visited their recently and loved it. :)
#68
This line is their of course.
I have an idea I"m going to try;
03/24/2010 (7:10 pm)
Dam I may have spoke too soon. Now I get an ignorable error that the CopyTo() method of my T2DStaticSprite needs the line obj2.Material = this.Material; This line is their of course.
I have an idea I"m going to try;
#69
It isn't a race condition as I had thought. When I unload the levels the T2DStaticSprites Dispose() Method is called.
My game send me to my gui.
Then when I click to start the level again the levels load and the error (or null reference if the checks are not in) happens before the Dispose() method is again called in the T2DStaticSprite.
So it seams that some of the objects that have been disposed from the last unload are being cloned when the level reloads.
Perhaps the Scene Container problem you have fixed is related?
The ignorable obj2.Material = this.Material; error is from the Torque.Util.TestObjectCopy.Test() Method.
03/24/2010 (8:23 pm)
Heya, Some info that might help.It isn't a race condition as I had thought. When I unload the levels the T2DStaticSprites Dispose() Method is called.
My game send me to my gui.
Then when I click to start the level again the levels load and the error (or null reference if the checks are not in) happens before the Dispose() method is again called in the T2DStaticSprite.
So it seams that some of the objects that have been disposed from the last unload are being cloned when the level reloads.
Perhaps the Scene Container problem you have fixed is related?
The ignorable obj2.Material = this.Material; error is from the Torque.Util.TestObjectCopy.Test() Method.
#70
03/25/2010 (8:12 am)
Ok, I think this can be wrapped with a few more coding. I hope to have put an end to it :) I'll send now the patch files.
#71
I upped my game to playtest: http://forums.xna.com/forums/t/50323.aspx
03/25/2010 (12:00 pm)
Pino I applied the changes and things are freaking fantastic! I upped my game to playtest: http://forums.xna.com/forums/t/50323.aspx
#72
It works perfectly.
I do have that error but finally found the problem. A change you made in the engine made the problem in my code easier to trace. I think its on my end as if I stop spawning objects in components that are referenced externally in code the error goes away.
I think my code is holding a component open somewhere. Once I track that down I should be ok. :)
We all owe you a big one!
(heya Henry going to try your game tonight :) )
03/25/2010 (8:24 pm)
You Rock so much!It works perfectly.
I do have that error but finally found the problem. A change you made in the engine made the problem in my code easier to trace. I think its on my end as if I stop spawning objects in components that are referenced externally in code the error goes away.
I think my code is holding a component open somewhere. Once I track that down I should be ok. :)
We all owe you a big one!
(heya Henry going to try your game tonight :) )
#73
I've sent the consolidated engine to John and Eric but had no answers from them till now. I really don't want to redo the whole work when they release a new version (and SVN merging can be nasty!) so I really hope they will import my changes int next version or I'm quite stuck to this revised 3.1.4. Fingers crossed.
03/26/2010 (1:18 am)
Ok Matthew, I suspected something about that but... one never knows :) Did you refer to the IsDisposed property? I now move to the TX3D library to consolidate that one to work with the new shared Core.I've sent the consolidated engine to John and Eric but had no answers from them till now. I really don't want to redo the whole work when they release a new version (and SVN merging can be nasty!) so I really hope they will import my changes int next version or I'm quite stuck to this revised 3.1.4. Fingers crossed.
#74
The last one I have to work out is that the first object that is spawned with a T2DSpawner object will be spawned while the quad is null. And the quad will only be null for the first object created (regardless of what object I create first) By disabling the T2DSpawner the error goes away (before it would error out on the component I was referencing in code).
The thing is all objects I create with the T2DSpawner I also reference in code. (most of the objects created are platform nodes and I reference them to create nodes in my A* path-finding system). I'm guessing the problem is that something is holding onto them so they can't dispose properly. I'm going to track this down this weekend. :)
Thanks again.
Also I would be quite upset if GG didn't put your changes into their next release. The community really shouldn't need fix these types of release stopping bugs. I'm glad we have the community we do. ;)
03/26/2010 (8:08 am)
Heya, I'm not sure exactly what I did to cause the issue yet. Still investigating. I know that I reference some of my components in code external to the torque architecture and found that they could not dispose. Setting those references to null fixed some of the errors.The last one I have to work out is that the first object that is spawned with a T2DSpawner object will be spawned while the quad is null. And the quad will only be null for the first object created (regardless of what object I create first) By disabling the T2DSpawner the error goes away (before it would error out on the component I was referencing in code).
The thing is all objects I create with the T2DSpawner I also reference in code. (most of the objects created are platform nodes and I reference them to create nodes in my A* path-finding system). I'm guessing the problem is that something is holding onto them so they can't dispose properly. I'm going to track this down this weekend. :)
Thanks again.
Also I would be quite upset if GG didn't put your changes into their next release. The community really shouldn't need fix these types of release stopping bugs. I'm glad we have the community we do. ;)
#75
03/26/2010 (7:34 pm)
Hi, I was wondering if I could get the SVN diff file for this also. Great job on hunting down all of the leaks! valknut13(at)gmail(dot)com Thank you.
#76
You can send it to this e-mail address:
john {at} suckerfreegames {.} com
Thanks a bunch for all the work you did.
03/26/2010 (11:34 pm)
I too am wondering if I could get that patch file. It would be a huge help for me.You can send it to this e-mail address:
john {at} suckerfreegames {.} com
Thanks a bunch for all the work you did.
#77
Ron just found a new leak that indeed was my fault (getting too old!)
Please comment out the line 1081 in T2DParticleEffect.cs (_effectData = null).
Thanks again Ron for finding this!!!
Pino
03/27/2010 (3:11 am)
Hi Guys,Ron just found a new leak that indeed was my fault (getting too old!)
Please comment out the line 1081 in T2DParticleEffect.cs (_effectData = null).
Thanks again Ron for finding this!!!
Pino
#78
03/27/2010 (3:28 am)
Wait a sec... there is a collateral effect: commenting that line when cloning from templates makes the particles stop working (at least in my game). I'll look into this a little more to understand what's going on. I'm really getting too old... I've to stop working too late :)
#79
Thanks so much for doing all this work! I was wondering if I could also get the patch file.
My email address is:
jonhoness(at)gmail(dot)com
Again, thank you for your hard work!
03/27/2010 (6:32 am)
Hi guys,Thanks so much for doing all this work! I was wondering if I could also get the patch file.
My email address is:
jonhoness(at)gmail(dot)com
Again, thank you for your hard work!
#80
My particle effects use a "KILL" effect mode. Perhaps this nullification needs to be conditional based on the effect mode.
Effectively my particle effect is a blood effect. So it happens once for each enemy kill. After each individual effect is played it is killed by the engine, then cloned again on the next enemy kill.
I have the following flags on my particle effect template:
X Pool
X Pool with components
X Template
X Persistent
Although, in retrospect, I guess "Pool with components" is not necessary since no components are attached to this template.
I'll continue to investigate using different effect modes and see what happens.
Thanks
--RB
03/27/2010 (6:47 am)
@Pino...I'm not seeing problems with the particle effects not working. The original symptom that lead me to the issue on line 1081 was that I was seeing an exponential growth in the number of particles being emitted. They seemed to double each cycle (presumably from not being cleared after the previous iteration).My particle effects use a "KILL" effect mode. Perhaps this nullification needs to be conditional based on the effect mode.
Effectively my particle effect is a blood effect. So it happens once for each enemy kill. After each individual effect is played it is killed by the engine, then cloned again on the next enemy kill.
I have the following flags on my particle effect template:
X Pool
X Pool with components
X Template
X Persistent
Although, in retrospect, I guess "Pool with components" is not necessary since no components are attached to this template.
I'll continue to investigate using different effect modes and see what happens.
Thanks
--RB
Associate Giuseppe De Francesco
DFT Games Ltd
in T2DStaticSprite.cs (line 43) change the Material property making it this way:
public RenderMaterial Material { // forward access to quad object get { if (_quad != null) return _quad.Material; else return null; } set { if (_quad != null) { _quad.Material = value; _quad.RefillVB = true; } } }This should really fix your issue allowing to properly dispose the RenderQuad.