Torque X huge memory leak - Fixed
by Giuseppe De Francesco · in Torque X 2D · 03/18/2010 (11:54 am) · 298 replies
Well,
I'm not sure this is the right place to post this, maybe the blog? At any rate I'm posting here now.
Torque X has a huge memory leak, but seriously! I was working to a RPG game for a start so I didn't really notice, but 2 weeks ago I started YASS, a multiplayer side scrolling space shooter: a lot of particles, really a lot, and a lot of projectiles as well.
To let you understand here follows a short video of the game:
This is the memory situation after 3 minutes of gameplay (single player!!!) the game slowed down a lot:
So, I searched the forum and I realized that this issue is an old one and GG never got a grip on it (go figure why...) so I decided to fix the problems myself because I really need to publish YASS ;)
Long story short: I revised the whole engine and made a lot of fixes, some of them really of great importance, so here it is the situation playing my game (as you can see after 9 minutes the game still is 20% less than the original 3 minutes!):
There are still many stuff to fix, but at any rate in the current state I'm able to publish the game ;) I will make the SVN diff files available upon request (free of charge: pino AT dftgames DOT com).
Cheers,
Pino
I'm not sure this is the right place to post this, maybe the blog? At any rate I'm posting here now.
Torque X has a huge memory leak, but seriously! I was working to a RPG game for a start so I didn't really notice, but 2 weeks ago I started YASS, a multiplayer side scrolling space shooter: a lot of particles, really a lot, and a lot of projectiles as well.
To let you understand here follows a short video of the game:
This is the memory situation after 3 minutes of gameplay (single player!!!) the game slowed down a lot:
So, I searched the forum and I realized that this issue is an old one and GG never got a grip on it (go figure why...) so I decided to fix the problems myself because I really need to publish YASS ;)
Long story short: I revised the whole engine and made a lot of fixes, some of them really of great importance, so here it is the situation playing my game (as you can see after 9 minutes the game still is 20% less than the original 3 minutes!):
There are still many stuff to fix, but at any rate in the current state I'm able to publish the game ;) I will make the SVN diff files available upon request (free of charge: pino AT dftgames DOT com).
Cheers,
Pino
About the author
In the software eng. field since 1981, in charge of R&D during last 10 years. IEEE Senior Member (and volunteer).
#182
@Tom: I don't have TX3D so I can't confirm it, but it ought to work just fine (it's in Torque Core, which is common to both the 2D and 3D engines). There is one small ammendment to the TX3D engine that Pino needs to make, but depending on your usage of scenes in your game, it may or may not prevent you from using the async loader already.
04/21/2010 (10:33 am)
@Ron: check out the SVN project's wiki for an example. I'll also be putting up a wiki page or something to describe some of the options and stuff like that. I was thinking of also starting a thread there too, where people can discuss the async loader (since only people who have access to the SVN can use it anyway - rather than clog up this gg forum thread with it).@Tom: I don't have TX3D so I can't confirm it, but it ought to work just fine (it's in Torque Core, which is common to both the 2D and 3D engines). There is one small ammendment to the TX3D engine that Pino needs to make, but depending on your usage of scenes in your game, it may or may not prevent you from using the async loader already.
#183
I can try out just your changes and see if they work. Do you rely on Pino's changes for your stuff to work?
04/21/2010 (10:40 am)
@Duncan: Since Pino has not yet finished the 3D stuff yet I am still basically on the stock TX engine (all parts) with a few small changes I made to changes some things just for my game. Can you tell me what the 3D change is that Pino needs to make since I will have to manually make it to mine?I can try out just your changes and see if they work. Do you rely on Pino's changes for your stuff to work?
#184
04/21/2010 (11:07 am)
I believe you will need to add the following to T3DSceneComponent. Pino will be able to confirm once he gets the time to check it out.public override void SetSceneGraph(BaseSceneGraph scenegraph)
{
base.SetSceneGraph(scenegraph);
SceneGraph = scenegraph;
}
#185
04/21/2010 (11:18 am)
Thanks. I will have to try it out. 
#186
I load from a gui there is no scene in memory. It's like this
There are a couple of other odd things that happened, like one of my particle effects are no longer attached to the spawn, and my hero ship's gun does not get mounted. I don't know if they are being ignored in the scenedata or what.
04/21/2010 (12:52 pm)
Need to add a line to TorqueBase, line 111 add: [TorqueCloneIgnore] for the SceneData.I load from a gui there is no scene in memory. It's like this
public class GuiLoadScreen : GUIBitmap, IGUIScreen
{
all the gui stuff.
_loadText.Text = "Loading:";
Game.Instance.Engine.GameTimeScale = 0; // pause the play
Game.Instance.AsyncSceneLoader.LoadScene(@"data\levels\mylevel.txscene", null, OnAsyncSceneLoaded, null);
}There are a couple of other odd things that happened, like one of my particle effects are no longer attached to the spawn, and my hero ship's gun does not get mounted. I don't know if they are being ignored in the scenedata or what.
#187
www.xp-dev.com/wiki/77988/Asynchronous%20Scene%20Loading%20In%20Torque%20X
04/21/2010 (2:48 pm)
New wiki entry on the SVN site:www.xp-dev.com/wiki/77988/Asynchronous%20Scene%20Loading%20In%20Torque%20X
#188
The async loader basically is a copy of the original loader code just spread out a bit. It's possible that some bugs got introduced in the spreading-out though. And it's also very possible that there are some further engine modifications required to handle the async loading.
I will make a new thread on the SVN site.
In the meantime -
Question: are the particle effect and the gun the only things that are mounted in the scene file?
04/21/2010 (2:55 pm)
That's interesting feedback Henry. I don't use the Torque GUI stuff so that's just the kind of testing need, thanks.The async loader basically is a copy of the original loader code just spread out a bit. It's possible that some bugs got introduced in the spreading-out though. And it's also very possible that there are some further engine modifications required to handle the async loading.
I will make a new thread on the SVN site.
In the meantime -
Question: are the particle effect and the gun the only things that are mounted in the scene file?
#190
:)
04/21/2010 (3:40 pm)
@Duncan thanks for the wiki and samples...looks like pretty good stuff.:)
#191
I will build a test file to share.
04/21/2010 (3:55 pm)
@duncan no if you have XBLIG you can see my game or check the video www.youtube.com/watch?v=nD9a6QzJTDM There you can breifly see the ships in space. One set of ships has the particle trail the other oes not. The particle trails are templated, mounted to templates. I traced through the code and the gun (T2DAnimation) does exist and apparently gets mounted. However does not show up.I will build a test file to share.
#192
04/21/2010 (4:27 pm)
Okay look forward to seeing the test (email is: duncan at flooge plus com).
#193
Thank you again for doing it.
04/21/2010 (4:33 pm)
Without getting into it too much it looks likes your async loading is mixed in with Pino's stuff a bit too much. I may have to wait until he gets his 3D stuff all done before I can use it. I will keep an eye on it, but will probably go to review without it. My loading is down to about 10 seconds so it isn't too bad.Thank you again for doing it.
#194
If you just get the Torque Core project then that will be enough (apart from the small mod to the 3D engine I mentioned earlier - as far as I know there aren't any other 3D mods required).
04/21/2010 (4:42 pm)
The async loader isn't reliant on Pino's memory fix stuff - it's 'mixed in' to the extent that it is in the same SVN project, but that's all. Also, the loader is in Torque Core which is common to the 2D and 3D engines.If you just get the Torque Core project then that will be enough (apart from the small mod to the 3D engine I mentioned earlier - as far as I know there aren't any other 3D mods required).
#195
04/21/2010 (4:45 pm)
Just had a thought - do you mean that the T3D does not work with Pino's modded Torque Core yet? if so then I see what you mean.
#196
When I say mixed in it is stuff like in Pino's changes there is a dispose method in TorqueObject. I don't have that in my version because I am not using Pino's stuff at all. So I am not sure which stuff I need to put into it and which I don't. I would have to go back and forth looking at my code versus pino's and then that versus yours. It would be very complicated to figure out at this point.
04/21/2010 (5:02 pm)
Yes, Pino's 3D does not work yet with the Core changes. It all compiles and runs, but the frame rate is about 1/2 frame per second. Pino is working on it, but has been busy lately.When I say mixed in it is stuff like in Pino's changes there is a dispose method in TorqueObject. I don't have that in my version because I am not using Pino's stuff at all. So I am not sure which stuff I need to put into it and which I don't. I would have to go back and forth looking at my code versus pino's and then that versus yours. It would be very complicated to figure out at this point.
#197
The main square of the terrain looks OK, but if you go higher where you can see around, the repeated terrain blocks are in primary colors, like blue and red etc.
04/21/2010 (5:24 pm)
I tried the new core with the 3D and it does something weird with the terrain, I was about to post an image, and then look into it however I had other stuff that was way more pressing. The main square of the terrain looks OK, but if you go higher where you can see around, the repeated terrain blocks are in primary colors, like blue and red etc.
#198
Things on our side are being arranged ( lawyers and paperwork, etc. ) so that we can move forward as aggressively as you guys are. Keep up the hard work! Soon we will be driving this so that you guys don't have to.
04/21/2010 (5:44 pm)
I've been following this thread and all I can say is "wow". You guys can really cut through the tape. Things on our side are being arranged ( lawyers and paperwork, etc. ) so that we can move forward as aggressively as you guys are. Keep up the hard work! Soon we will be driving this so that you guys don't have to.
#199
Guys,
About the TX3D: as mentioned above I'm having serious issues working on my project because I cannot use the 3D editor at all, so I have to rely on the FPS demo project alone which is quite reductive (I had to freeze my TX3D project a while ago because of the editor issue). I'll appreciate very much if any of you able to use the editor (maybe on XP? who knows!) will send me some unit test style projects for me to isolate the problems because (repetita iuvant!) I'm unable to create unit tests myself on any of my computers (I have only Win7 computers here).
There are two known issues till now:
1) when phisics is involved in the scene the frame rate drops to less than 10(!).
2) Henry's finding: I don't see that in the FPS Demo so a repro project would be great.
Thanks,
Pino
04/22/2010 (12:42 am)
@Eric: well that will be quite welcome on this end :) But please... as here we got it under control please focus on giving us as first priority (IMO) a new integrated editor: the TX3D one crashes 10 times a day!Guys,
About the TX3D: as mentioned above I'm having serious issues working on my project because I cannot use the 3D editor at all, so I have to rely on the FPS demo project alone which is quite reductive (I had to freeze my TX3D project a while ago because of the editor issue). I'll appreciate very much if any of you able to use the editor (maybe on XP? who knows!) will send me some unit test style projects for me to isolate the problems because (repetita iuvant!) I'm unable to create unit tests myself on any of my computers (I have only Win7 computers here).
There are two known issues till now:
1) when phisics is involved in the scene the frame rate drops to less than 10(!).
2) Henry's finding: I don't see that in the FPS Demo so a repro project would be great.
Thanks,
Pino
#200
04/22/2010 (5:21 am)
I don't have Windows 7, but is it possible to run an virtual XP machine? 
Torque Owner Tony Pitman
Default Studio Name