RAM issues

#1 You didn't mention what texture resolution you're using for your interiors. We have alot of interiors in our enviorment and I have yet to see such a big memory footprint. Maybe you should take a look at your interiors and think about reducing some of the details that you got and:

1. Make them into detail brushes
2. Create DTS's of them, thus making the collision meshes smaller total.
3. Reuse already used textures.
4. Lower the detail level

As you didn't mention how many brushes/polygons etc you're using for the interiors and how many you got placed there's no way anyone can say what the problem is without guessing. They could be awfully detailed and that's why it chokes.

We got around 30-35 interiors in one dense area and it doesn't kick up further than to 200 megabytes used. This is without optimizing the interiors used and with using 512x512 textures.

#2 We don't have as many interiors as yours (it's just some simple houses), and most our textures are 256x256 or lower, and most of them are being re-used already.

I was suspecting out lightmap resolutions, but the mission lighting file is just 2MB big.

Maybe it's got to do with our interiors, and the way they were compiled... is there a way to get a better overview of RAM usage inside Torque? Like, how much RAM is being used by each object?

#3 You can dump allocations (see Pat Wilson's excellent resource) - if you really want to know, I'd suggest spending an hour writing some code to query the memory usage before/after initialization steps.

#4 I would guess it's mission lighting... set $pref::Interior::VertexLighting to 1 and see if your ram usage plummets... I have yet to write the lightmap collapsing/sharing code we need for our game... it's a significant issue... our levels go from using 600-800 megs of ram to using less than 200 when using vertex lighting :|

#5 I'll test the vertexlighing and see if the lightmaps are the culprits. If so, then those .ML files use some m4d compression...

#6 Turning the lightmaps off reduced RAM usage to 110 MBs! That's 200MBs worth of lightmaps! Holy cow!

This *is* a significant issue. In what kinda format are those stored in RAM? The .ML file is only 2MB big!

I can't think of any solutions right now, since we're not in bug-fixing and optimization phase yet, but I'd like to start collecting suggestions already. I'd love to use vertex lighting, but we get this kind of weirdness everywhere:

#7 My guess is they are not compressed in the GPU memory. Just a wild guess though :) Glad you tracked down the problem.

Try some detail brushes to eliminate the artifacts shown in your picture. That's the problems you'll get when using vertex lighting though :/

#8 We've switched to vertex lighting while developing, otherwise the swap disc gets too much of a workout. Setting up the lightmaps as compressed textures only shaved off around 60 megs when I tested it.

Btw, there is another thread on this here

#9 Josh, that's the solution to my problem too! I had created a thread asking about it but didn't find a solution and hadn't seen your other thread. Thanks!

I'll list my findings in case anyone is interested. I created an empty mission and put 18 instances of 1 DIF building with 380 brushes and 2167 surfaces.

Here are my results:

Vertex Lighting Disabled
Starting RAM: 190MB
RAM after mission loaded: 490MB (this can easily go up even more)

Time to load mission (first time - no .ml file): 50 secs
Time to load mission (second time): 27 secs

Vertex Lighting Enabled
Starting RAM: 190MB
RAM after mission load (first time - no .ml file): 410MB
RAM after mission load (second time): 270MB!!!

Time to load mission (first time): 35 secs
Time to load mission (second time): 10 secs!!!

So to recap, with VL disabled 300MB were added whereas with VL enabled only 80MB were added!

Can anyone tell me what are the advantages and disadvantages of using VL? And just as an FYI, I won't have any sun or too much lighting in my game.

And by the way, using the same mission in TSE with VL disabled I had only 300MB used which means it does a much better job in memory management while with VL enabled, the engine crashed.

Nick

Edit: added times

#10 Hey guys,

I think I can clear up some of the questions, by explaining a little about Torque's interior lighting:


Torque's Lighting Schemes

Torque has two interior lighting schemes; vertex and light map lighting. Vertex lighting stores the lighting amount at each vertex, which creates a lighting effect similar to the lighting on DTS objects. Light mapping stores a per-surface texture that contains detailed lighting and shadowing information, this looks much better but creates a greater memory footprint. I recommend using light maps whenever possible, the added realism of the lighting detail far outweighs the memory usage.


Shared Light Maps

Torque alleviates memory usage by providing a type of light map sharing or instancing. As interiors instances are loaded all interiors of the same type (the same dif and detail level) are assigned the same shared light maps. During mission load all surfaces that are 'outside-visible', that is surfaces in zone 0 or in zones that allow ambient lighting to enter the zone (check out the portal entity for details), are updated with ambient and directional lighting information making the light maps unique thus creating a duplicate light map. Surfaces that are in a zone that does not allow ambient lighting through are not duplicated, saving a lot of memory.

The best way to avoid a large light map memory footprint is to make sure all interiors have at least two zones (inside and outside), also make sure the portals do not allow ambient lighting through.

Even more importantly (shameless plug ;) the Lighting Pack allows you to edit the light map scale (the size of the surface light maps) on individual interior brushes. If you're using the Lighting Pack make sure all exterior brushes have small light map sizes, keeping the duplicated light maps as small as possible. If you are not using the Lighting Pack you'll be forced to lower the entire interior's light map scale to reduce memory.


Light Map Border Sizes

A while back Torque had a problem related to light maps and texture filtering that caused all kind of strange colors to appear on interiors (like this pink line):


In order to fix this I added borders to Torque's interior light maps (this is in stock TGE as of version 1.3). I tweaked the border size until the lines disappeared, however when using anti-aliasing the lines didn't completely go away until the border size was 10 pixels (ouch!), this was due to the AA filtering on some video cards. When using large light maps the added border is not that big of a deal, however the border size has a huge impact on small light maps, especially when there are a lot of them. This increases the light map memory usage considerably. Oh and TSE seems to do a better job of memory management because it doesn't have this code yet, internally TGE and TSE have the same light map sharing mechanism.

My recommendation is to tweak the border size to your liking (the macro is 'SG_LIGHTMAP_BORDER_SIZE' in map2dif's 'lmapPacker.h', you'll need to recompile the interior you're working on for the change to have an effect), just be aware of the potential artifacts that this tweak is preventing.

Another option is to disable mipmapping on light maps. The colored line problem is related to a combination of bleeding in the lower mipmap levels and filtering. The code I've added helps solve most of the filtering problem, but the filtering problem is exaggerated by bleeding in the sampled down light maps. Removing the mipmapping will also create some visual artifacts, but they may not be as bad or noticeable at all, so it's worth a shot.



Anyway this is just a quick overview of Torque's light mapping, hopefully it covers a lot of the questions. Let me know if this helps!

-John

#11 John, thats some very insightfull and useful info!!

Personally though, I'll probably end up using vertex lighting anyway because I can't use any portals with my interiors (most of them are ruins with no roof and many open windows). This will probably be of bigger help to Josh Ritter's problem.

I will try your recommendations though and see what I come up with.

Nick

#12 John,

We have our interior portals setup correctly. We've also scaled the lightmaps to the point that they are almost looking like vertex lighting... Vertex lighting isn't really an option as it's fugly and full of lighting errors.

I will try disabling lightmap mipmaps, compressing the lightmaps, and changing the border size.

Thanks,
-Josh

#13 Alright,

1. It appears that lightmaps don't have mipmaps, somewhat oddly they use: BitmapNoDownloadTexture ... which carrries through to their their GL texture creation...

2. Changing the border size to 1 has a HUGE impact on ram usage. We recompiled one section of Tim Aste's Castle to test this, just the inside of the castle. RAM usage went down by 60 megs... I expect similar ram savings on the other pieces.

3. Changing the border size to 1 brings back the colorful artifacting you mention.

Given the ram usage, I don't think changing the border size is a fix for the color artifacts.


-Josh

EDIT: Just dropped a city mission 700 megs to 330 megs by recompiling some difs with bordersize to 1 and using texture compression. It'll go down further by recompiling all the difs without the bordersize. The color artifacting now needs to be solved, two steps forward and one back.

#14 Thanks for the answers! I'll fight against discarding our lightmaps until the very end!

There's something very odd in this whole thing... the videocard I'm using has 128MBs of RAM. If the lightmaps were using that much VRAM, the framerates would've gotten worse with lightmaps on, since VRAM would be exhausted and system RAM would be used instead. But the framerates are the same with and without lightmaps, what leds me to think that the actual textures are not using that much VRAM (or Torque is smartly managing VRAM).

Whatever Torque is doing, the lightmaps are stored in system RAM with no compression whatsoever, and this might explain the massive RAM usage.

How are the lightmaps stored in the .ML file? It's so small...

#15 They use PNG which uses zlib...

One thing, the color of the artifacts has no basis in any lighting present in the map/mission file... last I checked we have no pure green lights... ;)

Here is some color artifacting when the border is set to 10:


The artifacting vanishes when the border is set to 1

See above for the effect the border size has on ram usage.

-J

#16

Quote:But the framerates are the same with and without lightmaps, what leds me to think that the actual textures are not using that much VRAM

But it could also just mean that you're bottlenecked somewhere else - in geometry processing or on the CPU, perhaps.

#17 The funky color artifacts are coming from what appears to be some edge debugging code in lmapPacker.cc:

m_sheets.last().pData = new GBitmap(csm_sheetSize, csm_sheetSize);
   for (U32 y = 0; y < m_sheets.last().pData->getHeight(); y++) 
   {
      for (U32 x = 0; x < m_sheets.last().pData->getWidth(); x++) 
      {
         U8* pDst = m_sheets.last().pData->getAddress(x, y);
         pDst[0] = 0xFF;
         pDst[1] = 0x00;
         pDst[2] = 0xFF;
      }
   }

and

m_sheets.last().pData = new GBitmap(in_sizeX, in_sizeY);
   for (U32 y = 0; y < m_sheets.last().pData->getHeight(); y++) 
   {
      for (U32 x = 0; x < m_sheets.last().pData->getWidth(); x++) 
      {
         U8* pDst = m_sheets.last().pData->getAddress(x, y);

         if (overflow == false) 
         {
            pDst[0] = 0xFF;
            pDst[1] = 0x00;
            pDst[2] = 0xFF;
         }
         else 
         {
            pDst[0] = 0x00;
            pDst[1] = 0xFF;
            pDst[2] = 0x00;
         }
      }
   }

When using texture compression these become very obvious...

-Josh

#18 By dropping the border size to 0 and using texture compression (which required removing the edge colors from the lightmap packer) one of the cities went from using 700+ megs to using 150 megs ... that's pretty darn good.

-Josh

#19 I tried altering the color of the dead space before, but ultimately it's a static color and will still create artifacts depending on the lighting in the light maps. Unfortunately a light map border, even a small one, is necessary to avoid artifacts.

I'll take a look at the light map textures later, it seems odd that they're not using mipmaps.

#20 Right, I would rather the artifact be black than green or purple as it's a tad more believable.

A border setting of 10 gets rid of much artifacting while blowing memory usage through the roof. A border of 0 (or 1) keeps memory happy and makes for tons of lighting errors.

I am sure there's a way to have the cake and eat it too...

-Josh