TX 3.1.5: SceneContainerBinReference leak (with proposed fix)
by Michael Russell · in Torque X 2D · 02/13/2011 (6:12 am) · 11 replies
Build: 3.1.5 Release
Platform: All
Target: PC, Xbox 360
Issues: Every time a new scene is loaded, the previous SceneContainerBinReference[] array in SceneContainer leaks.
Steps to Repeat:
1. Install VS 2008 Team Developer
2. Add a starter 2D game.
3. In the starter 2D game, add an action to unload the current level and load a new level.
4. Instrument the game with allocations and lifetime
5. During instrumentation, trigger the action repeatedly.
6. Look at the allocation reports. You'll find one instance of SceneContainerBinReference[] allocated and live at the end, regardless of the number of garbage collections.
Suggested Fix:
Due to the size of the objects, they are going into the Large Object Heap and as such the .NET Garbage Collector is treating it differently. This fix has been submitted to the 4.0 CEV for evaluation, but only three files need to be modified.
File fixes as replies.
Platform: All
Target: PC, Xbox 360
Issues: Every time a new scene is loaded, the previous SceneContainerBinReference[] array in SceneContainer leaks.
Steps to Repeat:
1. Install VS 2008 Team Developer
2. Add a starter 2D game.
3. In the starter 2D game, add an action to unload the current level and load a new level.
4. Instrument the game with allocations and lifetime
5. During instrumentation, trigger the action repeatedly.
6. Look at the allocation reports. You'll find one instance of SceneContainerBinReference[] allocated and live at the end, regardless of the number of garbage collections.
Suggested Fix:
Due to the size of the objects, they are going into the Large Object Heap and as such the .NET Garbage Collector is treating it differently. This fix has been submitted to the 4.0 CEV for evaluation, but only three files need to be modified.
File fixes as replies.
About the author
Former QA Manager at Ritual Entertainment, Software Test Lead at Microsoft Game Studios.
Recent Threads
#2
On line 394, change the class declaration to:
Add the following implementation:
02/13/2011 (6:17 am)
\SceneGraph\SceneGraph.csOn line 394, change the class declaration to:
public abstract class BaseSceneGraph : TorqueObject, IDisposable
Add the following implementation:
#region IDisposable Members
void IDisposable.Dispose()
{
if (Container != null) Container.Dispose();
base.Dispose();
}
#endregion
#3
At the top of FindObjects (line 417), add a line to early out if disposed because something still ends up calling the object after Dispose() for one frame.
GetNumActiveReferences (line 467): Rewrite to get rid of foreach loop
GetNumObjects (line 497): Rewrite to get rid of foreach loop
_FindObjectsInefficiently (line 669)
_GetObjects (line 695)
BinArray caching next reply
02/13/2011 (6:27 am)
\SceneGraph\SceneContainer.csAt the top of FindObjects (line 417), add a line to early out if disposed because something still ends up calling the object after Dispose() for one frame.
public void FindObjects(SceneContainerQueryData queryData)
{
if (IsDisposed) return; // <== this line gets added
// Find which bins are covered.
uint minX, minY, maxX, maxY;GetNumActiveReferences (line 467): Rewrite to get rid of foreach loop
public int GetNumActiveReferences()
{
int total = 0;
int sceneBinArrayCount = _sceneBinArray.GetUpperBound(0) + 1;
for (int i = 0; i < sceneBinArrayCount; i++) // <== Replace foreach with this
{
SceneContainerBinReference walk = _sceneBinArray[i]; // <== Replace binRef with array dereference
while (walk != null)GetNumObjects (line 497): Rewrite to get rid of foreach loop
public int GetNumObjects()
{
Dictionary<ISceneContainerObject, bool> objs = new Dictionary<ISceneContainerObject, bool>();
int sceneBinArrayCount = _sceneBinArray.GetUpperBound(0) + 1;
for (int i = 0; i < sceneBinArrayCount; i++) // <== Get rid of foreach loop
{
SceneContainerBinReference walk = _sceneBinArray[i]; // <== Change binRef to array dereference
while (walk != null)_FindObjectsInefficiently (line 669)
protected internal int _FindObjectsInefficiently(SceneContainerQueryData query)
{
int objectsFound = 0;
_currentSequenceKey++;
int sceneBinArrayCount = _sceneBinArray.GetUpperBound(0) + 1;
for (int i = 0; i < sceneBinArrayCount; i++)
if (_sceneBinArray[i] != null)
_FindObjectsInefficientHelper(query, _sceneBinArray[i], ref objectsFound);
_FindObjectsInefficientHelper(query, _sceneOverflowBin, ref objectsFound);
return objectsFound;
}_GetObjects (line 695)
protected internal void _GetObjects(List<ISceneObject> list)
{
_currentSequenceKey++;
int sceneBinArrayCount = _sceneBinArray.GetUpperBound(0) + 1;
for (int i = 0; i < sceneBinArrayCount; i++)
if (_sceneBinArray[i] != null)
_GetObjectsHelper(_sceneBinArray[i], list);
_GetObjectsHelper(_sceneOverflowBin, list);
}BinArray caching next reply
#4
Add a private struct and static list to hold the bin cache
Update _CreateBins (line 705) (note: this version isn't threadsafe)
One last change to Dispose() (line 926)
02/13/2011 (6:33 am)
Allocate an int reference for the new bin cacheprivate int _binIndex = -1;
Add a private struct and static list to hold the bin cache
private struct SceneBinArrayHolder
{
public bool IsAllocated;
public uint BinCount;
public SceneContainerBinReference[] SceneBin;
}
static List<SceneBinArrayHolder> SceneBinArrays = new List<SceneBinArrayHolder>(2);Update _CreateBins (line 705) (note: this version isn't threadsafe)
protected void _CreateBins()
{
// Create Scene Bin Array.
// <== Changes start here
_binIndex = -1;
for (int i = 0; i < SceneBinArrays.Count; i++)
{
if (!SceneBinArrays[i].IsAllocated && SceneBinArrays[i].BinCount == _binCount)
{
_binIndex = i;
break;
}
}
if (_binIndex == -1)
{
_binIndex = SceneBinArrays.Count;
SceneBinArrays.Add(new SceneBinArrayHolder());
}
SceneBinArrayHolder hold = SceneBinArrays[_binIndex]; // To work around CS1612
hold.IsAllocated = true;
if (hold.SceneBin == null)
{
hold.SceneBin = new SceneContainerBinReference[_binCount * _binCount];
hold.BinCount = _binCount;
}
else
{
for (int i = 0; i < _binCount * _binCount; i++)
{
hold.SceneBin[i] = null;
}
}
SceneBinArrays[_binIndex] = hold;
_sceneBinArray = hold.SceneBin;
// <== Changes end here
// if configured to do so, create individual array objects.
// this can end up creating a lot of objects.
if (PreAllocSceneBinArrayReferences)
for (int i = 0; i < _sceneBinArray.Length; ++i)
_sceneBinArray[i] = new SceneContainerBinReference();One last change to Dispose() (line 926)
public override void Dispose()
{
_IsDisposed = true;
for (uint i = 0; i < SceneBinArrays[_binIndex].SceneBin.Length; i++)
{
if (SceneBinArrays[_binIndex].SceneBin[i] != null)
SceneBinArrays[_binIndex].SceneBin[i].Reset(); // Just to make sure to not create a circular reference of disposed objects
SceneBinArrays[_binIndex].SceneBin[i] = null;
}
_ResetRefs();
SceneBinArrayHolder hold = SceneBinArrays[_binIndex]; // To work around CS1612
hold.IsAllocated = false;
SceneBinArrays[_binIndex] = hold;
if (_sceneOverflowBin != null)
{
_sceneOverflowBin.Reset(); // Just to make sure to not create a circular reference of disposed objects
_sceneOverflowBin = null;
}
_sceneBinArray = null;
if (_sg != null)
{
_sg.Reset();
_sg = null;
}
}
#5
Great job! Please commit these changes to the CEV repositories so that the users can have this and report back ;)
02/13/2011 (2:24 pm)
Hey Michael,Great job! Please commit these changes to the CEV repositories so that the users can have this and report back ;)
#6
02/14/2011 (1:01 pm)
Awesome! We were noticing this just a bit if we reloaded FishCraft levels repeatedly (never was an issue but may have been hitting performance a bit). We will give this a test and let you know how it goes.
#7
Hey, I really appreciate you posting this. I had noticed a problem in our game where if we reloaded the level a few times the Xbox would run out of memory, and I bet this is it. I'll have to do more testing later, but thanks again for looking into it!
One thing I noticed is that I was getting a crash with your changes on reloading a level. Turns out it was because I was cloning an object that had a mounted, pooled particle effect on it. The pooled particle effect still had a reference to the old scenegraph, which had a container that contained a null _sceneBinArray, which was causing problems.
To fix this, I added this line to the end of OnUnregister() in T2DSceneObject:
This should make sure that the pooled objects don't point to an old scenegraph. Let me know if this is a bad idea or if I'm doing something wrong.
Thanks again!
02/20/2011 (9:40 am)
Michael,Hey, I really appreciate you posting this. I had noticed a problem in our game where if we reloaded the level a few times the Xbox would run out of memory, and I bet this is it. I'll have to do more testing later, but thanks again for looking into it!
One thing I noticed is that I was getting a crash with your changes on reloading a level. Turns out it was because I was cloning an object that had a mounted, pooled particle effect on it. The pooled particle effect still had a reference to the old scenegraph, which had a container that contained a null _sceneBinArray, which was causing problems.
To fix this, I added this line to the end of OnUnregister() in T2DSceneObject:
_sceneGraph = null;
This should make sure that the pooled objects don't point to an old scenegraph. Let me know if this is a bad idea or if I'm doing something wrong.
Thanks again!
#8
To be honest, I'm not sure if that's a bad idea or not yet. I'm a recent acquirer of TX, and I've found a good way to learn codebases is to try to track down bugs other people have reported, which is how I'm approaching it.
Memory leaks were something I excelled at tracking down in my previous professional career, so I started with this one.
I'll research it and check it.
02/20/2011 (9:58 am)
Jacob,To be honest, I'm not sure if that's a bad idea or not yet. I'm a recent acquirer of TX, and I've found a good way to learn codebases is to try to track down bugs other people have reported, which is how I'm approaching it.
Memory leaks were something I excelled at tracking down in my previous professional career, so I started with this one.
I'll research it and check it.
#9
Glad to have your skills looking into this :)
I wanted to set up your test with my code, but I don't have the Team Developer version of Visual Studio 2010 (I use Express). I see that there is a standalone profiler download for VS 2008, but I'm not sure if that works with 2010.
I don't suppose you (or anyone else reading this thread) knows of a good way for VS2010 Express users to profile for memory leaks in a similar manner?
Thanks!
02/20/2011 (10:28 am)
Michael,Glad to have your skills looking into this :)
I wanted to set up your test with my code, but I don't have the Team Developer version of Visual Studio 2010 (I use Express). I see that there is a standalone profiler download for VS 2008, but I'm not sure if that works with 2010.
I don't suppose you (or anyone else reading this thread) knows of a good way for VS2010 Express users to profile for memory leaks in a similar manner?
Thanks!
#10
http://blogs.msdn.com/b/davbr/archive/2011/02/01/clrprofiler-v4-released.aspx
02/20/2011 (10:53 am)
CLR Profiler v4 was released back on February 1.http://blogs.msdn.com/b/davbr/archive/2011/02/01/clrprofiler-v4-released.aspx
#11
02/20/2011 (11:05 am)
Thanks, I'll give that a shot. I didn't know a new version had been released!
Torque Owner Michael Russell
RomSteady Software
Rearrange Unload() slightly to prevent a NullReferenceException.
public void Unload() { Assert.Fatal(_loaded, "TorqueSceneData.Unload - Level not loaded, Load it first."); if (!_loaded) return; _loaded = false; // we fire the callback first so that the recipient can unhook itself from content if (this.OnUnloaded != null) this.OnUnloaded(); // MR 2/12/11 - Moving before unregister to stop nullrefexception // now walk our lists and Unload stuff _UnloadObjects(this.SceneData); _UnloadObjects(this.Materials); _UnloadObjects(this.Objects); if (_levelFolder != null && _levelFolder.IsRegistered) { // unregister the torque folder. This should unregister everything in it, which will include all objects that were registered // since the time the level was created. TorqueObjectDatabase.Instance.Unregister(_levelFolder); _levelFolder = null; // we null things out for garbage collector } this.SceneData = null; this.Materials = null; this.Objects = null; }