Oops, using a lot of RAM

#21 No, I'm not using tile maps. I am using particles. My debug build is around 120 Mb when running in comparision to 87 Mb for the shipping build.

#22

Quote:T2D - TGB has alpha, beta, "early adopter" stamped all over it! what's unclear about that?

You know that I know this as well as anyone. Still... when a little warning blurb at the end says it's ea after saying that it is the best 2d game engine in the world... come on. It gives the impression that it's not feature complete but that what it has is already to use.

I support GG and Torque engines more fiercely than anyone, but I was having a hard time this time and I am asking for help to help keep GG and TGB looking like the shit. I really appreciate the help so far too. I am shouting Torque from the rooftops and I am just pointing out an error and being honest about both the impact on the game and the impression that it gives to the company. I am not talking about some hobby dickaround project either. I am being completely on TGBs side... but I'm also in a bind here with it.

You know, look up angry rants about torque and you'll find me calming the pissed off party down. This isn't an angry rant. I am completely understanding of the situation. I am just asking for help from you guys. My biggest worry here is that someone wouldn't take the time to help on this when they know how to fix it. Every game that releases on TGB means more licenses sold and more games releasing.

You guys gave me food for thought above. Any more ideas are welcome.

The game is using the latest TGB.
It has some very large images files, way beyond 256x256. Most with widths between 4000-20000.
I am wondering if the image splitting process is where the memory bloat is happening.

#23 @Anton sorry dude- I understand your frustration and I wasn't questioning your commitment to the platform. :-)

#24 Many of the images are not powers of 2 either.

Question... what is t2d doing when it splits up the larger images? Are their temp images being created?
And is t2d fixing the non-powers of 2 images and making copies in powers of 2? And... is there temp images being made during PNG decompression? Could something like temp images being left in memory be the cause of this bloat? Cause this is 1.2 gb of RAM not virtual ram. This won't run on my machince with only 768mb RAM. It was dev'd on machines with 2gb RAM.

Thanks for the help. :)

#25 Are you really sure the problem is graphics, and not sound or music? I've seen two problems recently, one for both sound and music where leaks were happening and getting out of control.

For perf reasons, you should seriously consider making your images a more "reasonable" size and using POT where possible. Why depend on image split-up code in a production game? That code is there for noobies that just want to get something working. If you have a game that is "near production," you have to optimize these things out.

#26 I'll just jump in and describe quickly the packing process that TGB uses. When TGB determines that an image needs to be packed, it simply identifies the frames using the current image-map mode. This equates to creating a list of rectangular regions in a list (not copying any image-map data).

The "packing" process is two-fold. The first is to wrangle this rectangular region list into a set of textures that are compatible with the current hardware or the current global overrides. This first stage does consume memory but it doesn't copy image data, it's just a tree of rectangular regions fitting into virtual spaces.

When the "packing" process is happy that it has met the criteria for maximum packing or least textures as selected, it then begins the process of generating the new texture page(s) and copying the original image-data into these new textures.

It then destroys the original texture.

During this process, you will have memory consumption for the quad-tree and a subsequent allocation for the new textures followed by the removal of the original texture.

All this happens fairly quickly and is pretty efficient. I certainly wouldn't call it memory bloat considering image sizes of 4000-20000 are being dealt with.

In terms of tracking where your allocations are being made; I'd suggest using the wealth of fine-grained detail provided by the image-map system to the scripts in terms of callbacks/echos that you can turn-on. This stuff is detailed in the advanced section at the end of the image-map doco.

Dependant upon where your textures are being loaded, could you not look at the allocation before/after the image-map allocations?

I tend to agree with Jason in that the "packing" process, whilst powerful, was intended as a "just work" for people who don't want to be bothered with optimization issues of image construction.

I would really be interested in working with you to identify any potential areas where I could reduce the memory footprint of the packing process, especially if you've essentially got a test case for a stress-test with so large a volume of imagery.

We could do that here or we could take it privately if you prefer. We can then post back here on our results?

Sound cool?

- Melv.

#27 @Melv

The programmer made headway on it already. I let him know about what you said. He only has 10 days to finish up some other stuff so he won't be able to go thru anything with you on this, but he really appreciated the offer. Which was extremely generous of you by the way. Anything that I can glean off of this from him that helps you I am more than happy to relay to you. I'm crazy busy too, so, if it takes a bit to get anything about that on here, that's why. I'm sure you understand that. :)

Thanks for working so hard!

Anton

#28 No problem, I'm here to help.

- Melv.

#29 The first thing, if not done already it to make sure your games structure/data loading i efficient. By this i mean, if you have basically 4 sections to your game (splash,menu,gameplay and credits) make sure you have seperate datablocks. For the splash (if you have one) it should not only be unloaded, but before you set the canvas to the next gui you should destroy it, wont be needed again for this run.

Also note that TGB 1.4 datablock has a "preload" property that defaults to TRUE and a "allowUnload" which defaults to FALSE. Try swapping these, dont preload and allowUnload.

Finally, image basics, this is not required for TGB but I always suggest. First, make sure imahe width andheight as divisable by 2. Next All you little images bullets, whatever, try o try to get down into 16 colors? or 256 will always look just fine.

If all these are done, The end result should be the most optimized memory footprint your game will get. Little late, but thought I'd post my process for all anyway.

#30 Erm .... The color advice is useless I fear.
Modern cards internally only support 16 or even 32bit ... so even a 2 bit image will end with 32bit.
Such tricks are only usefull to save disk space / download bandwidth ...

#31 Ahh, did not know that since the last few years game programmers have not been forced to do that. But as you say, if you are making a big game with alot of images, going to 256 can at least save you quite a bit of bandwidth / storage space.

#32 I just built an ran the FishDemo from the TGB Release samples, and while it's running it's consuming about 96Mb of RAM on my system as well... Considering that the content of this is Very small... I cannot see why it would be using so much RAM...

Any useful information on how to reduce the RAM usage would be very useful, cause if the FishDemo runs at 96Mb and does nothing but display fish swimming around, I'm assuming that any useful game would be easily 3 -4 times this much...

Don't get me wrong, I know RAM is cheap, but hey... I woundn't complain if it were like 20 Mb of RAM ...

Thanks.

I have tried to run with the -notools as well.. saved 0 Mb... no diff... At least it doesn't seem to be growing, at least for the 5 min that I ran it and watched...

#33 Scott, you'll note that this thread is a few months old. There were a few memory problems when we were going through the 1.1 ALPHA phases, when this thread was created. They have since been cleaned up. And note as well that Melv helped Anton work through the problems his particular project was experiencing.

As for the Fish Demo... 96MB is not much, and it actually incorrect to extrapolate that any game would automatically consume 3-4 times as much memory. Though the fish imagery is small in terms of disk size, since its compressed, when its loaded into memory its fairly large. We chose to use some pretty high-res images, so people could view the fish demo at very high resolutions without things looking stretched and blurry. We also have large splash / demo screens that are just big graphics. And there are quite a few objects on the screen at once, as you can see if you turn on debug rendering. Simple games actually wouldn't have much going on, or require as many memory resources, as this demo.

With the entire level editor and the whole default project loaded, TGB only eats up 80MB of RAM on my machines (a desktop, and a laptop). Depending on your system and configurations, it might eat up to 120MB or so... but of course remember that the level editor and the default scene have *lots* of resources to load. Most games won't even need that much.

If you just load the TGB executable, it takes about 10MB of RAM... which is normal for most any Windows app. If you load the TGB executable and start it up with a blank scene, execute the typical common scripts, and do nothing else, you're eating 12-30MB of RAM, depending on your machine. Its not a memory hog by any stretch, and in fact those numbers are better than other (much less featureful) game SDKs out there.

To note as well, we have a TGB game in development internally here. It is very much a casual game, and its system requirements will be quite low-spec, even though its a very high-end, graphically rich casual game. For this game, we did not had to dig in and tweak anything about the low-level ways in which TGB loads or handles memory itself. But we did have to be sure that in the game code itself, we were being smart about the way we load resources. That's not because of anything in particular with TGB... rather, any time you load resources from disk, or have a really rich set of graphics images (as with our game), you have to be careful about how you load them. Loading from disk any time you need an image or resource might cause a visible hitch, since the disk needs to be read from, and that is slow. On the other hand, pre-loading all your images and resources can cause long start-up times and result in sustained overhead memory usage.

The cool thing about TGB is we have a really nice system for handling graphics resources, that lets you absolutely control how and when images should be loaded and unloaded. And if you don't want to dink with it, the system will try to do a decent job of figuring things out on its own. The other cool thing is that this is all really well documented, in the ImageMap reference documentation... it a pretty technical doc, but that's because this is pretty complicated, important, technical stuff.

Hope this was helpful!