Torque X huge memory leak - Fixed

#1 Hi Pino...it's Chapel from Creators Club.

I would love to see those SVN diffs.

If you don't want to post them here...can you please email them to me?

Let me know if you do not have my email address.

BTW - YASS looks great...I'll look around for it to review on CC.

Thanks
--RB

#2 This is very similar to my issues lately, I'd love to check out your files. I am really excited that you have overcome the issues. Like I said in my post, there are lots and lots of bullets not to mention enemies and of course tons of particles. The memory profiler showed large memory use by the particles however i never had that big of a slowdown, just jumping every few seconds or so. Thanks Pino!

bigdaddio at smokinskull.com

#3

Quote:So, I searched the forum and I realized that this issue is an old one and GG never got a grip on it

Is there a main reason for this issue happening or is it a lot of little things all over the engine?

#4 I'd love to see the svn diff files too if you could.
Email hoesterey@gmail.com

Thanks for finding these issues. :)

#5 @Duncan: it's a ton of omitted finalizations, almost nothing is properly disposed in the engine, a lot of root referencing never cleaned up.

I cannot imagine how such a code has passed the testing stage and has been allowed to be delivered to the market. Maybe the team has been reshuffled a few times during the development... I don't know. today I'll generate the diffs and email them to you guys.

EDIT: The files have been sent.

#6 Sounds like you've done a great job on this! I've sent you an email.

#7

Quote:it's a ton of omitted finalizations, almost nothing is properly disposed in the engine, a lot of root referencing never cleaned up.

This is interesting - I'd love a copy of the diffs. You can email me at "duncan@flooge [dot] plus [dot] com"

Thanks

#8 Well looks like it's good for the PC version but not enough for the poor Xbox. I'm now able to play SEVEN whole minutes (!) on the console before getting a Code 4.
But now the code 4 is not because of the memory allocation, now it's all about resource allocation (no more handles!) so ... let's hit the "revise the engine" road again... I knew that I didn't had 100% sorted... I'll post again in a while when all is smoothly done (hopefully tonight!).

#9 Now since I am not a SVN expert how do I apply the diff to the file?

#10 Resource allocation: GPU resources I'm assuming?

#11 Henry, I post here some SVN info:

Well first off you have to install a local SVN server, which is quite simple. Get it from www.collab.net/downloads/subversion/

Then get Tortoise SVN from here: tortoisesvn.net/downloads and install it.

I’m not sure if the VS2008 SVN plug-in works with the Express version, give it a try: ankhsvn.open.collab.net/servlets/ProjectProcess?pageID=3794

Once on is on just add the 3.1.4 original to a new repository. If the VS2008 plug-in works with Express you can do that from VS: open the Engine solution (C:\Program Files\GarageGames\Torque X 2D Pro 3.1.4\v3.1.4.0\TorqueEngine.sln), right click on it the solution explorer. In the contextual menu you should have “Add Solution to Subversion”.
If that isn’t working then you have to use Tortoise: open the source installation folder (C:\Program Files\GarageGames\Torque X 2D Pro 3.1.4\) then right click on the v3.1.4.0 folder and in the contextual menu choose Tortoise->Export. This will add the whole folder’s content to your SVN server, so after you might want to use the Tortoise->Repo-Browser to remove from the repository the unwanted files. That done add them to the Ignore List.

Having the above done, simply right click on the project folder (TorqueCore and then Torque2D) and select Tortoise->Apply Patch: it’ll ask you for the diff file and it’ll take care of the patching.

#12 I would also like to get these patch files when your fix your resource problem.

you can e-mail them to me at blackpanther(AT)panthers-den(dot)com

#13 Thanks Pino I got it figured out, I used Tortise Merge, I have an svn set up here but really I just think of it as a backup.

This looks good, I am pretty excited.

#14 Hi guys,

I think I got rid of almost all the resource allocation problems, in fact I can now play my game till the end on the Xbox without any code 4!

Here's the current situation:



As you can see now the Core is allocating and freeing object in the same way of the rest of the revised engine. Anyway before to say that's fixed I call it a night, sleep on it and in the morning I'll double check the new changes to see if I missed anything, then I'll email you the patches to put on top of the first ones.

#15 Runs fantastic, seriously this was a big issue, my GC latency time no longer grows but remains at a pretty steady 20ms every 3-4 seconds. When I use the memory profiler T2DParticleEffect is not longer the largest thing in memory and I have only 10-15 effects in memory at once not 100+. Great Job Pino.

I have run into a couple small issues

ResourceManager.cs Line: 217

/// <summary>
/// Returns the resource object.
/// </summary>
public T Instance
{
    get { return _resource.Instance; }
}
[code]

This value is null when calling GUISplashScreen (probably other places as well) I reverted it to the original code.

T2DParticleEffect.cs Line: 1444

[code]
this.SceneGraph.RemoveObject(this);

I changed to this:

if (this.SceneGraph != null)
    this.SceneGraph.RemoveObject(this);

I have a pause screen where the player can exit to the main menu. I guess if there are effects in memory or whatever that were not yet registered or whatever but anyway this.SceneGraph was null.

I also ran on the xbox for way more than 7 minutes with no issues whatsoever. This really made a large difference in my code, thanks for the effort Pino. Maybe you should send John a copy?


[/code]

#16 Hi HEnry,

we posted at the same time :) I'm glad that this comes useful to you!

I haven't tested the GUI scenario because I don't use it, so any double check on that is very welcome!

I'm puzzled... I can't find that line of code in the ResourceManager.cs, my code is this one:

/// <summary>
        /// Returns the resource object.
        /// </summary>
        public T Instance
        {
            get
            {
                if (_resource != null)
                    return _resource.Instance;
                else
                    return default(T);
            }
        }

Can you double check plese?

#17 Henry, I don't have John's email address :) if you do, ping him telling to have a look to this thread.

#18 Heya, I implemented your changes. Everything seams to run great and the game is using less memory!


Few issues though.

Scene Change Issue:

Game.Instance.SceneLoader.UnloadLastScene(); crashes the game with a "mis-match between object and cookie" error

It looks like its having problems when a T2DSceneObjects tries to Dispose itself and throws the above error.

I then get a Null reference error in the T2DSpawnObject.cs in the Dispose method on this line

_sceneGraph.RemoveObject(this);

Particle Effect Issue:

One Line you added

_effectData = null;

in T2DParticleEffect actually causes multiple copies of the effect to be created over time. ie.. the first time the effect plays one copy plays. The 2nd 2 copies, the 3rd 3 ect.. removing this line fixes the problem, though I don't know if it causes others.


Getting a few minor bumps but other then that the code seams to work great.

ps. I haven't been able to contact John through email lately. Our emails have trouble sometimes though :(.

And Henry I'm not seeing that code in the resource manager either :(.

#19 I must have madea mistake like I said thsi is the first time I have merged in code from a diff file. ; )

Weird! I am looking right at the diff file and the code I got is not whats in the diff, like I said I must have screwed something up, but weirdly how?

#20 @Henry: I guess that the issue is caused because you used Merge instead of Apply Patch: AFAIK they differ quite a lot.

@Matthew: thanks for the feedback. The _effectData thing makes sense because the effect is based in the XNA SimpleEffect so it's fully manages by the ContentManager, so I didn't have to put that line :) The NullReferences are caused by a timing issue I guess, so I've now put those lines after a null check.

I'm about to send to you guys the new patch files. They are differential so are supposed to go on top of the first set.

Here all checks out quite well (no errors raised in any situation I tested: scene changing, intensive spawining, intense collisions, intense particle spawinig and killing): let me know if there are issues with your setting so I can fix them.