Radical reduction in TGB memory usage. Yay!
by Jon Frisby · 05/28/2007 (8:25 pm) · 19 comments
UPDATE: This patch does NOT work when switching between full-screen/window, or presumably when changing screen resolutions. In fact, sadly, the engine crashes. The short form is that it's not liking having to go to disk to fetch texture data for textures where it doesn't have a name (chunked images for example -- such as fonts?).
The "right" way to address this seemed to be changing t2dImageMapDatablock to specify a texture type of BitmapTexture, but that doesn't work for the same reason -- and passing in the mSrcBitmapName so the texture has a name only works if the image type is "FULL". For CELL or KEY images, you get wackiness because of the preprocessing of the image data performed by t2dImageMapDatablock::loadTextures not being performed by the texture manager. So Tim and I are discussing a more thorough fix, and hopefully we'll have something to post next weekend or so.
In the meantime, ONLY use this patch if your game doesn't allow changing resolutions or switching between full-screen/window!
---
I recently discovered that the reason the Deluxe version of Harmonic Convergence was taking 560MB of RAM was... Texture data. Yep. I should've noticed this earlier, but had higher priorities. Easy to miss that sort of thing when your dev box has 2GB of RAM.
Making use of the proper tools (notably $pref::T2D::imageMapShowPacking) revealed that I was losing several hundred MB of RAM to padding. In some cases I had stupidly set filter padding on images that were powers of two (effectively quadrupling their memory usage). In some cases I found images that were very high resolution, compared to the amount of screen real estate they were being allotted. If an image will never occupy more than a 20 pixel by 20 pixel area of the screen, then there's no point in the image being 160x160. In a number of cases the artists had produced backgrounds or layers for the native screen resolution (800x600 -- which pads out to 1024x1024). One 1024x1024 texture should take 4MB of memory. Now multiply that several times for different layers. Now multiply it by two because a 1024x1024 texture actually winds up taking 8MB in TGB.
Uh. What?!
It's not MIP levels (a la TGE), it's just that TGB is keeping the original bitmap data around after creating the texture object. Ouch!
So after having spent a huge amount of time editing art assets -- scaling/shrinking, clipping dead whitespace and slowly accumulating some savings (I got as far as cutting memory usage by just over 100MB!), Tim-MGT sent me a two line patch that eliminated the needless double-memory-usage, and it seems to work without any obvious "gotchas" (like crashes).
In gTexManager.cc, find the function whose declaration reads:
Go down to the bottom of this method, and add this code:
Right before the line that reads:
Instant reduction in memory usage.
-JF
The "right" way to address this seemed to be changing t2dImageMapDatablock to specify a texture type of BitmapTexture, but that doesn't work for the same reason -- and passing in the mSrcBitmapName so the texture has a name only works if the image type is "FULL". For CELL or KEY images, you get wackiness because of the preprocessing of the image data performed by t2dImageMapDatablock::loadTextures not being performed by the texture manager. So Tim and I are discussing a more thorough fix, and hopefully we'll have something to post next weekend or so.
In the meantime, ONLY use this patch if your game doesn't allow changing resolutions or switching between full-screen/window!
---
I recently discovered that the reason the Deluxe version of Harmonic Convergence was taking 560MB of RAM was... Texture data. Yep. I should've noticed this earlier, but had higher priorities. Easy to miss that sort of thing when your dev box has 2GB of RAM.
Making use of the proper tools (notably $pref::T2D::imageMapShowPacking) revealed that I was losing several hundred MB of RAM to padding. In some cases I had stupidly set filter padding on images that were powers of two (effectively quadrupling their memory usage). In some cases I found images that were very high resolution, compared to the amount of screen real estate they were being allotted. If an image will never occupy more than a 20 pixel by 20 pixel area of the screen, then there's no point in the image being 160x160. In a number of cases the artists had produced backgrounds or layers for the native screen resolution (800x600 -- which pads out to 1024x1024). One 1024x1024 texture should take 4MB of memory. Now multiply that several times for different layers. Now multiply it by two because a 1024x1024 texture actually winds up taking 8MB in TGB.
Uh. What?!
It's not MIP levels (a la TGE), it's just that TGB is keeping the original bitmap data around after creating the texture object. Ouch!
So after having spent a huge amount of time editing art assets -- scaling/shrinking, clipping dead whitespace and slowly accumulating some savings (I got as far as cutting memory usage by just over 100MB!), Tim-MGT sent me a two line patch that eliminated the needless double-memory-usage, and it seems to work without any obvious "gotchas" (like crashes).
In gTexManager.cc, find the function whose declaration reads:
TextureObject* TextureManager::registerTexture(const char* textureName, const GBitmap* data, bool clampToEdge)This should be in the vicinity of line 1094 if you're using TGB 1.5 beta 3.
Go down to the bottom of this method, and add this code:
delete ret->bitmap; ret->bitmap = NULL;
Right before the line that reads:
return ret;
Instant reduction in memory usage.
-JF
#2
Somebody tested this on the 1.1.3?
05/29/2007 (3:08 am)
Great. Sometimes the simples solution (2 lines) gives the most benefits. (not having to re-edit all the artwork)Somebody tested this on the 1.1.3?
#3
05/29/2007 (3:19 am)
In case anyone missed it... Make sure you read the update at the top of the page, it turned out to not be quite as simple as expected. I'm sure Jon and Tim will come up with a more robust solution though, so stay tuned. 
#4
05/29/2007 (3:35 am)
Nice stuff Magnus, hope your able to address the drawbacks of the solution too :) 
#5
That way anything that wants or needs to keep the bitmap around, will still do so.
EDIT: Just re-read your update and it seems you've thought of this already :)
05/29/2007 (5:47 am)
Have a look at where TGB loads the textures in. There should be a TextureHandle call which you can then pass in a type of BitmapTexture or BitmapKeepTexture. If you pass BitmapTexture instead of BitmapKeepTexture (or not type at all) then you call the overridden registerTexture method which includes the "delete" and "null" lines you've posted. That way anything that wants or needs to keep the bitmap around, will still do so.
EDIT: Just re-read your update and it seems you've thought of this already :)
#6
What you've mentioned above is correct and I noticed the overhead that the "mSrcBitmap" (base Source Bitmap from disk) was taking so I remove that as soon as the packing is complete. With that out the way, the only things that are left are the "mVecTexturePage" texture pages. Each of these elements contains a structure that describes a complete texture page. It references the dynamic bitmap / texture handle (and other metrics) that was created during the packing process (don't forget the original bimap has gone now).
The problem (from my notes) was that during texture resurrection "void TextureManager::resurrect()", the texture manager needs the bitmap. Now texture resurrection can happen any time you change resolution or toggle full-screen. Here's the problem; in the resurrection, if the bitmap is null (for the associated texture), the texture manager will attempt to load the associate bitmap from its name and expects it to be there. T2D dynamically creates these textures during packing so they don't exist on disk, hence the name being registered as NULL and the bitmap (of type "RegisteredTexture") is kept so that during resurrection a new texture is created from it.
To get around this I had a note to amend a few items...
The code in "t2dImageMapDataBlock.cc" that generates the texture with default type of "RegisteredTexture" can use the overload that allows you to specify the type to something other like "BitmapTexture".
This way the bitmap is destroyed after the texture object is created. Now the problem is that during resurrection, having this texture type requires that you have a disk file name for the texture and of course it doesn't exit. The only way I can think of quickly to stop this is to make a minor modification to the resurrection routine to gracefully ignore textures that are of this type and don't have file names. You can skip the line in the routine...
So this would stop the crash but it obviously leaves the texture missing. The way then to get around this is to take advantage of the final call in the resurrection routine...
The texture manager lets you register/unregister callbacks for zombification/resurrection. You can see it being used in "gDynamicTexture.cc"...
What you can do in the t2dImageMapDatablock during "OnAdd" and "OnRemove" is register/unregister with the texture manager. In the callback, you simply have to call "bool t2dImageMapDatablock::compileImageMap( void )".
In theory the above methods (maybe with refinements) would cause a recalculation of the dynamic textures that TGB uses and immediately remove them during texture-object creation. Obviously the downside is that the packing process has to be done again. With that said, you can easily modify it so that you reuse the packing information by making a new call such as "recompileImageMap()" that reuses the existing packing calculations and simply generates the appropriate bitmaps which should save considerable time dependent upon the bitmaps in question.
I hope this helps a little. Please bear in mind that most of this is from notes so there may be other modifications needed.
Good Luck!
- Melv.
05/29/2007 (1:38 pm)
I've not looked at that code for a while now but I did have a quick look through my old folder notes for you (I've got a folder for almost every class in T2D) and on the "TODO" page are a few notes.What you've mentioned above is correct and I noticed the overhead that the "mSrcBitmap" (base Source Bitmap from disk) was taking so I remove that as soon as the packing is complete. With that out the way, the only things that are left are the "mVecTexturePage" texture pages. Each of these elements contains a structure that describes a complete texture page. It references the dynamic bitmap / texture handle (and other metrics) that was created during the packing process (don't forget the original bimap has gone now).
The problem (from my notes) was that during texture resurrection "void TextureManager::resurrect()", the texture manager needs the bitmap. Now texture resurrection can happen any time you change resolution or toggle full-screen. Here's the problem; in the resurrection, if the bitmap is null (for the associated texture), the texture manager will attempt to load the associate bitmap from its name and expects it to be there. T2D dynamically creates these textures during packing so they don't exist on disk, hence the name being registered as NULL and the bitmap (of type "RegisteredTexture") is kept so that during resurrection a new texture is created from it.
To get around this I had a note to amend a few items...
The code in "t2dImageMapDataBlock.cc" that generates the texture with default type of "RegisteredTexture" can use the overload that allows you to specify the type to something other like "BitmapTexture".
// Generate Texture.
texturePage.mpPageTextureHandle = new TextureHandle( NULL, texturePage.mpPageBitmap, true );
// to...
texturePage.mpPageTextureHandle = new TextureHandle( NULL, texturePage.mpPageBitmap, BitmapTexture, true );
texturePage.mpPageBitmap = NULL;This way the bitmap is destroyed after the texture object is created. Now the problem is that during resurrection, having this texture type requires that you have a disk file name for the texture and of course it doesn't exit. The only way I can think of quickly to stop this is to make a minor modification to the resurrection routine to gracefully ignore textures that are of this type and don't have file names. You can skip the line in the routine...
GBitmap *bmp = loadBitmapInstance(probe->texFileName);... if the condition is met but you'd have to quickly remove the texture node which is easy.
So this would stop the crash but it obviously leaves the texture missing. The way then to get around this is to take advantage of the final call in the resurrection routine...
postTextureEvent(CacheResurrected);
The texture manager lets you register/unregister callbacks for zombification/resurrection. You can see it being used in "gDynamicTexture.cc"...
mTexCBHandle = TextureManager::registerEventCallback(dynamicTextureCB, this);
What you can do in the t2dImageMapDatablock during "OnAdd" and "OnRemove" is register/unregister with the texture manager. In the callback, you simply have to call "bool t2dImageMapDatablock::compileImageMap( void )".
In theory the above methods (maybe with refinements) would cause a recalculation of the dynamic textures that TGB uses and immediately remove them during texture-object creation. Obviously the downside is that the packing process has to be done again. With that said, you can easily modify it so that you reuse the packing information by making a new call such as "recompileImageMap()" that reuses the existing packing calculations and simply generates the appropriate bitmaps which should save considerable time dependent upon the bitmaps in question.
I hope this helps a little. Please bear in mind that most of this is from notes so there may be other modifications needed.
Good Luck!
- Melv.
#7
www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=10656
tdn.garagegames.com/wiki/2D_Art_Tips:_Learn_From_My_Mistakes
Melv: any chance we could see an optional parameter for this in the standard source tree so that we wouldn't have to all have custom implementations that may end up becoming incompatible?
Thanks!
-Andrew
05/29/2007 (5:44 pm)
Two things to help reduce memory utilization:www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=10656
tdn.garagegames.com/wiki/2D_Art_Tips:_Learn_From_My_Mistakes
Melv: any chance we could see an optional parameter for this in the standard source tree so that we wouldn't have to all have custom implementations that may end up becoming incompatible?
Thanks!
-Andrew
#8
06/05/2007 (8:55 am)
@Andrew: Saw those already. In fact, I spent quite a lot of time optimizing images to avoid texture wasteage. However, 2x 200MB (the approximate total image size, with no waste) is still 400MB so killing the doubling of memory is a pretty critical issue for me.
#9
06/05/2007 (8:56 am)
@Melv: Thanks for the tips! I will make good use of them!
#10
We've been wrestling with what seems like RAM over-usage for a long time in TGB 1.7.4 and prior. One current game uses a large number of textures through its course, and the RAM quickly blows up to above the onboard RAM of most video cards, leading to hitches and other performance issues on lesser video cards. Compared to games of similar graphics intensity, the utility Process Explorer shows TGB to be using much more RAM. It hovers in the 300MB plus range for 3 1024x768 backgrounds and a couple dozen small pieces. Comparably other games of this scope usually are in the 100MB to 150MB range.
Our techniques for optimization:
1. We're using imageMapDumpTextures=1 and being sure everything's packed as well as it can be
2. We're using the tricks in www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=1065 in other words, preload=0, allowUnload=1 for most/all datablocks, then using a dynamic load of the sets we need, and a dynamic unload also. This helps quite a bit over preload=1, but the overall RAM still creeps up a lot.
3. Each area has its own datablock set that includes background, pieces, objects relevant to the scene, and we load 3 scenes at a time (a stage). We exec the datablocks, then do a dynamic image load each stage, and and unload of the prior.
The specific problems we're hitting:
1. Exec'ing the datablocks, even though preload=0 for everything, runs RAM up by 200+ meg from the baseline.
2. Haven't found a way to successfully unload RAM between stages to make physical RAM usage drop (as seen in Process Explorer). Would like to unload all the stuff from stage 1 when loading stage 2 to keep a smaller RAM footprint and keep more in physical memory rather than virtual.
Questions:
1. For games with a lot of graphics, is there a built-in way to unload RAM (eg. where Process Explorer will show a notable drop?). flushTextureCache and other such methods didn't seem to do it. It seems to bump against an upper limit of the video card and stay there, with a lot of paging occurring.
2. Is there a way to unload datablocks after they've been declared and return the physical/virtual RAM? Like maybe using a SimSet or something?
3. Have these engine mods (the ones posted by Melv) been used successfully by anyone to halve RAM usage? If so are there any bad side effects (game will need to go fullscreen)?
We love TGB and the RAM issue is one that is hurting us performance-wise on the lower end PCs as we make larger and larger games. Thanks for any tips here!
01/07/2009 (12:46 pm)
These posts sound very interesting, we are looking at modding the engine to see if they work. It's an older post, not sure it it's in TGB 1.7.4 or not? I'll assume not for this description... checked the engine code and it didn't seem to include it.We've been wrestling with what seems like RAM over-usage for a long time in TGB 1.7.4 and prior. One current game uses a large number of textures through its course, and the RAM quickly blows up to above the onboard RAM of most video cards, leading to hitches and other performance issues on lesser video cards. Compared to games of similar graphics intensity, the utility Process Explorer shows TGB to be using much more RAM. It hovers in the 300MB plus range for 3 1024x768 backgrounds and a couple dozen small pieces. Comparably other games of this scope usually are in the 100MB to 150MB range.
Our techniques for optimization:
1. We're using imageMapDumpTextures=1 and being sure everything's packed as well as it can be
2. We're using the tricks in www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=1065 in other words, preload=0, allowUnload=1 for most/all datablocks, then using a dynamic load of the sets we need, and a dynamic unload also. This helps quite a bit over preload=1, but the overall RAM still creeps up a lot.
3. Each area has its own datablock set that includes background, pieces, objects relevant to the scene, and we load 3 scenes at a time (a stage). We exec the datablocks, then do a dynamic image load each stage, and and unload of the prior.
The specific problems we're hitting:
1. Exec'ing the datablocks, even though preload=0 for everything, runs RAM up by 200+ meg from the baseline.
2. Haven't found a way to successfully unload RAM between stages to make physical RAM usage drop (as seen in Process Explorer). Would like to unload all the stuff from stage 1 when loading stage 2 to keep a smaller RAM footprint and keep more in physical memory rather than virtual.
Questions:
1. For games with a lot of graphics, is there a built-in way to unload RAM (eg. where Process Explorer will show a notable drop?). flushTextureCache and other such methods didn't seem to do it. It seems to bump against an upper limit of the video card and stay there, with a lot of paging occurring.
2. Is there a way to unload datablocks after they've been declared and return the physical/virtual RAM? Like maybe using a SimSet or something?
3. Have these engine mods (the ones posted by Melv) been used successfully by anyone to halve RAM usage? If so are there any bad side effects (game will need to go fullscreen)?
We love TGB and the RAM issue is one that is hurting us performance-wise on the lower end PCs as we make larger and larger games. Thanks for any tips here!
#11
I have just sent you a private email using the address in your profile. Let's see what we can do and then perhaps post the results here.
Melv.
01/07/2009 (2:43 pm)
Kalle,I have just sent you a private email using the address in your profile. Let's see what we can do and then perhaps post the results here.
Melv.
#12
Thank you tremendously for helping me to implement the memory reduction code. So far so good, it appears to have reduced the footprint of a memory intensive game by about 70-80%. We have a game in QA with the new engine to test the solution more thoroughly, and I'll post here if we uncover any issues related to this implementation.
Kalle
02/03/2009 (9:53 pm)
Melv,Thank you tremendously for helping me to implement the memory reduction code. So far so good, it appears to have reduced the footprint of a memory intensive game by about 70-80%. We have a game in QA with the new engine to test the solution more thoroughly, and I'll post here if we uncover any issues related to this implementation.
Kalle
#13
Future image-maps will work in a similar way so this memory reduction will be standard from now on.
Melv.
02/03/2009 (11:38 pm)
That's great news! Sorry I didn't get to the problem sooner.Future image-maps will work in a similar way so this memory reduction will be standard from now on.
Melv.
#14
would you mind sharing some information if it was as easy as it looks in Melv's post or you've found some glitches?
We're using intensively D3D as our rendered - due to problems with OpenGL under some Vista drivers (Intel, ATI/AMD). Have you checked if it still works with DirectX?
08/26/2009 (11:56 am)
Hi,would you mind sharing some information if it was as easy as it looks in Melv's post or you've found some glitches?
We're using intensively D3D as our rendered - due to problems with OpenGL under some Vista drivers (Intel, ATI/AMD). Have you checked if it still works with DirectX?
#15
Since then, many people have used this and have found several issues, all of which were solved.
The new Torque 2D doesn't use any of this code, it's redundant.
08/26/2009 (12:03 pm)
It's unrelated to OpenGL or DirectX. All it really does is remove the original source bitmap from memory. It has to reload it and reprocess it if the texture manager needs to resurrect textures (switching full-screen etc).Since then, many people have used this and have found several issues, all of which were solved.
The new Torque 2D doesn't use any of this code, it's redundant.
#16
Does anyone have full solution? We do not even know when would T2D appear and we have two projects running (it seems that they will be finished way before T2D reaches final stage).
I've checked on our released project if this helps and memory usage reduced by 15% (we spent at lot of time optimizing assets and the way we were loading them).
For projects not optimized and still in production we've got 40% improvement though - so I think it's worth implementing. Where can we find some information about issues not solved here but mentioned by you?
08/26/2009 (4:08 pm)
OK thanks Melv for quick reaction!Does anyone have full solution? We do not even know when would T2D appear and we have two projects running (it seems that they will be finished way before T2D reaches final stage).
I've checked on our released project if this helps and memory usage reduced by 15% (we spent at lot of time optimizing assets and the way we were loading them).
For projects not optimized and still in production we've got 40% improvement though - so I think it's worth implementing. Where can we find some information about issues not solved here but mentioned by you?
#17
"Since then, many people have used this and have found several issues, all of which were solved." From Melv's post 2 boxes up.
My current .exe changelist# is somewhere around 350 so I worry about integrating to a new version of TGB or switching to Torque 2D. Something I could plug into 1.7.4 would be ideal.
Back-story:
So... I am about 1 year in on a 2 man project using TGB 1.7.4. (My Partner and I had enough of big companies and left to build this game)
Everything has gone really well and I am really happy with TGB. We are just getting into the final stages of the project where optimization is becoming a priority.
Last night I realized the memory usage for the game was 1.3GB (Yes Gigabytes) I am in the process of fixing the filter padded pow2 textures, turning off preload/on allowUnload for 90% of textures etc... That got my usage down to 650megs. That is what brought me here, I was surprised to have 650 megs of mem being used when nothing should really be loaded. So now I know what is happening and have an nugget of an idea of what to do to fix it, but I would LOVE to have a roadmap of the gotchas and fixes that have already been discovered.
My partner and I kind of bit off more than we could chew with this project and have been living off our savings for a year while we work. Anything to speed up my resolution of a problem like this is really appreciated.
Thanks for any info.
07/02/2010 (10:24 am)
I am hoping I can get some information about the solution used in this thread. Melv/Kalle have the info I require I think. I am planning on implementing what is listed in this article, but I would love a heads up on the following:"Since then, many people have used this and have found several issues, all of which were solved." From Melv's post 2 boxes up.
My current .exe changelist# is somewhere around 350 so I worry about integrating to a new version of TGB or switching to Torque 2D. Something I could plug into 1.7.4 would be ideal.
Back-story:
So... I am about 1 year in on a 2 man project using TGB 1.7.4. (My Partner and I had enough of big companies and left to build this game)
Everything has gone really well and I am really happy with TGB. We are just getting into the final stages of the project where optimization is becoming a priority.
Last night I realized the memory usage for the game was 1.3GB (Yes Gigabytes) I am in the process of fixing the filter padded pow2 textures, turning off preload/on allowUnload for 90% of textures etc... That got my usage down to 650megs. That is what brought me here, I was surprised to have 650 megs of mem being used when nothing should really be loaded. So now I know what is happening and have an nugget of an idea of what to do to fix it, but I would LOVE to have a roadmap of the gotchas and fixes that have already been discovered.
My partner and I kind of bit off more than we could chew with this project and have been living off our savings for a year while we work. Anything to speed up my resolution of a problem like this is really appreciated.
Thanks for any info.
#18
07/02/2010 (12:03 pm)
why don't you merge your code with the 1.7.5 and thus have your project brought into the new version? wouldn't that be easier?
#19
I didn't realize that 1.7.5 had this fixed. Guess I need to RTFM. I am cherry picking the texture fix now. Will probably grab the open AL upgrade too.
Wish me luck.
I am not brave enough to merge the two projects fully as 1.7.4 is a known entity to me after a year of working/fiddling with it 60 hours a week.
Thanks for the heads up.
07/02/2010 (1:29 pm)
Good point Luis.I didn't realize that 1.7.5 had this fixed. Guess I need to RTFM. I am cherry picking the texture fix now. Will probably grab the open AL upgrade too.
Wish me luck.
I am not brave enough to merge the two projects fully as 1.7.4 is a known entity to me after a year of working/fiddling with it 60 hours a week.
Thanks for the heads up.
Associate David Higgins
DPHCoders.com