How Torque handles VRAM?
by Vern Jensen · in Torque Game Builder · 03/31/2008 (12:15 pm) · 12 replies
I've hinted at this question in previous posts, but they were about other topics, and I think too complicated, so they got no answers.
I'd simply like to know how Torque handles images in VRAM versus main memory. It would be very helpful to know:
1) Are all textures loaded into both main memory and VRAM? Or are only certain ones put into VRAM, depending on what is currently needed in the current level? (My guess is the latter: that ALL textures are loaded into main memory, if "preload" is checked, but not all of those are put into VRAM -- that they're uploaded to VRAM as they're needed only.)
2) I've heard about mipmaps, if I remember the name correctly, where a texture is scaled down half its size, half its size again, and so on, and all of these versions are put into VRAM. When does this happen? Every time a texture is put into VRAM? Would it be hard to modify Torque to eliminate creation of these mipmaps.
3) When a texture is marked "Allow unload," does this result in it being unloaded from VRAM as needed, or does this refer to main memory unloading too?
I'd simply like to know how Torque handles images in VRAM versus main memory. It would be very helpful to know:
1) Are all textures loaded into both main memory and VRAM? Or are only certain ones put into VRAM, depending on what is currently needed in the current level? (My guess is the latter: that ALL textures are loaded into main memory, if "preload" is checked, but not all of those are put into VRAM -- that they're uploaded to VRAM as they're needed only.)
2) I've heard about mipmaps, if I remember the name correctly, where a texture is scaled down half its size, half its size again, and so on, and all of these versions are put into VRAM. When does this happen? Every time a texture is put into VRAM? Would it be hard to modify Torque to eliminate creation of these mipmaps.
3) When a texture is marked "Allow unload," does this result in it being unloaded from VRAM as needed, or does this refer to main memory unloading too?
#2
End users have no direct control over what happens in VRAM space, except for scene design (not using dozens/hundreds of textures in the same scene, etc.). If I recall correctly, preload/unload only matters with Torque memory space, not VRAM.
Eliminating mipmaps is a really bad idea from a performance perspective. Why would you want to?
Here's an example of why it's a bad idea:
Most artists/developers want to have really pretty textures, so they commonly greatly over-do the resolution for textures. For example, 128x128, 256x256, and I've even seen 1024x1024 textures mapped to shapes that only actually take up very small total pixels (for reference, set your design resolution to 800x600, then set your shape's size of a test object to 256x256. You'll see it take up roughly 1/4th of your total screen.)
Mipmaps allow the video card to apply the most appropriate texture size for the actual amount of pixels the texture covers. Removing mipmaps would effectively be trying to cram huge amounts of textures up on to your video card that are completely un-needed (you can't squeeze 1024x1024 into a 50 pixel by 50 pixel screen area, etc.).
03/31/2008 (1:46 pm)
It's a complicated topic, but you've basically hit it: Torque keeps textures in normal memory, and pushes them to the video card as required, which then caches internally as appropriate.End users have no direct control over what happens in VRAM space, except for scene design (not using dozens/hundreds of textures in the same scene, etc.). If I recall correctly, preload/unload only matters with Torque memory space, not VRAM.
Eliminating mipmaps is a really bad idea from a performance perspective. Why would you want to?
Here's an example of why it's a bad idea:
Most artists/developers want to have really pretty textures, so they commonly greatly over-do the resolution for textures. For example, 128x128, 256x256, and I've even seen 1024x1024 textures mapped to shapes that only actually take up very small total pixels (for reference, set your design resolution to 800x600, then set your shape's size of a test object to 256x256. You'll see it take up roughly 1/4th of your total screen.)
Mipmaps allow the video card to apply the most appropriate texture size for the actual amount of pixels the texture covers. Removing mipmaps would effectively be trying to cram huge amounts of textures up on to your video card that are completely un-needed (you can't squeeze 1024x1024 into a 50 pixel by 50 pixel screen area, etc.).
#3
Given that the answers to your questions are fairly straightforward.
All textures are loaded into system memory and given to the driver. As previously stated, the driver will push textures to the card as needed. Thus, your guess is mostly correct, except it is the driver that uploads textures into VRAM, not Torque.
Torque generates and loads mipmaps on texture creation. When a texture is pushed to the card its entire mipmap chain is also pushed on. It is possible to disable mipmaps to reduce VRAM usage, but you will undoubtedly increase the number of texture cache misses which will hurt performance far more than any VRAM savings will help.
When Torque 'unloads' a texture it is deleted from system memory and the internal OpenGL texture is also deleted. If the OpenGL texture currently occupies VRAM then the driver will mark it as available. Regardless of how the texture is marked the driver will always unload it from VRAM as needed.
03/31/2008 (2:35 pm)
When a texture is first loaded by Torque it is also passed off to the OpenGL driver, and when a texture is 'unloaded' or deleted by Torque the internal OpenGL texture is also deleted. The OpenGL driver determines when the texture will be loaded into VRAM and how long it will stay. Drivers will generally only push textures to the card as needed and will unload the least recently used texture(s) if more VRAM is required.Given that the answers to your questions are fairly straightforward.
Quote:1) Are all textures loaded into both main memory and VRAM? Or are only certain ones put into VRAM, depending on what is currently needed in the current level? (My guess is the latter: that ALL textures are loaded into main memory, if "preload" is checked, but not all of those are put into VRAM -- that they're uploaded to VRAM as they're needed only.)
All textures are loaded into system memory and given to the driver. As previously stated, the driver will push textures to the card as needed. Thus, your guess is mostly correct, except it is the driver that uploads textures into VRAM, not Torque.
Quote:2) I've heard about mipmaps, if I remember the name correctly, where a texture is scaled down half its size, half its size again, and so on, and all of these versions are put into VRAM. When does this happen? Every time a texture is put into VRAM? Would it be hard to modify Torque to eliminate creation of these mipmaps.
Torque generates and loads mipmaps on texture creation. When a texture is pushed to the card its entire mipmap chain is also pushed on. It is possible to disable mipmaps to reduce VRAM usage, but you will undoubtedly increase the number of texture cache misses which will hurt performance far more than any VRAM savings will help.
Quote:3) When a texture is marked "Allow unload," does this result in it being unloaded from VRAM as needed, or does this refer to main memory unloading too?
When Torque 'unloads' a texture it is deleted from system memory and the internal OpenGL texture is also deleted. If the OpenGL texture currently occupies VRAM then the driver will mark it as available. Regardless of how the texture is marked the driver will always unload it from VRAM as needed.
#4
03/31/2008 (2:38 pm)
From the true expert himself :) 
#5
03/31/2008 (3:25 pm)
Thanks guiding me out of the dark ages, guys! I still think its all witchcraft though.
#6
Stephen wrote:
Unlike the examples you gave, I've been very careful to resize my textures as I go to ensure they're very close to what the actual size in-game is. It's certainly much closer to actual size than the closest power-of-two-smaller mipmap would be. There are obvious exceptions, such as if I were to do real-time scaling, but my game scales only one of my many sprites in real-time, and only to about 20% larger, and when larger, mipmaps wouldn't be of any benefit.
The one exception in my case would be for lower resolutions, such as 800x600, since I'm targeting 1680x1050 when I size my textures (I want the game to look good even on 20" or larger monitors). Such a large native resolution is part of the reason I'm running low on VRAM if other programs are also taking up VRAM on the card. But in any case, the point is that my textures *would* be larger than normal for 800x600, but otherwise, they're optimized to not need mipmaps, if the user is running on a larger display.
Alex wrote:
Alex, could you elaborate a little bit on that? What you mean by texture cache misses? From my limited understanding, I'd guess that it would be a fillrate problem, not a texture miss problem... that is, using a texture optimized for 1680x1050 in a 800x600 window, would require a certain amount of scaling down on-the-fly without any mipmaps... I'm not sure why there would be texture cache misses? (which I would understand to mean textures not being in VRAM, and therefore needing to be loaded from main memory to VRAM before the object can be drawn) It seems like I'd have fewer of those, since without the mipmaps taking up VRAM, I could have all of my objects in VRAM, versus most of them, hopefully eliminating misses entirely. Fillrate would suffer on 800x600 somewhat, but if the game can run full-speed on 1680x1050, I'd think the fillrate should be about the same if something is scaled down to a lower res, although that's just a guess. (i.e. it's *reading* the same number of pixels per frame as it would on 1680x1050, but writing fewer, and since it's done in hardware, I'd guess the speed would be about the same for both scenarios... i.e. reading 1000 pixels and writing 1000 should be similar to reading 1000 and writing only 500. Again I'm guessing though.) Although if that's such a problem, maybe I'd add some code to the engine to scale down my textures on-the-fly when I first load them, once, if a lower-than-intended resolution is used by the game.
If I'm still being stupid, and really should keep mipmaps, let me know. But otherwise, if a game's textures are intelligently optimized for a specific resolution, and almost never scaled smaller than their original size (or if scaled smaller, it's only small amounts, like 70% to 90% -- something that would never result in a smaller mipmap being used anyway), then could it be beneficial to remove the mipmaps? Or am I still not understanding something?
My impression is that the fillrate problems without mipmaps might be overexaggerated? Would drawing a 256x256 texture to a 16x16 area really take any *longer* than drawing it to a 256x256 area? If not, I don't think it's a problem for my specific scenario. For most users, yes it would be, but for me no.
-Vern
04/01/2008 (12:13 am)
Thanks so much for the replies, guys! Alex, I really appreciate the explanation of how things are working internally.Stephen wrote:
Quote:Eliminating mipmaps is a really bad idea from a performance perspective. Why would you want to?
Unlike the examples you gave, I've been very careful to resize my textures as I go to ensure they're very close to what the actual size in-game is. It's certainly much closer to actual size than the closest power-of-two-smaller mipmap would be. There are obvious exceptions, such as if I were to do real-time scaling, but my game scales only one of my many sprites in real-time, and only to about 20% larger, and when larger, mipmaps wouldn't be of any benefit.
The one exception in my case would be for lower resolutions, such as 800x600, since I'm targeting 1680x1050 when I size my textures (I want the game to look good even on 20" or larger monitors). Such a large native resolution is part of the reason I'm running low on VRAM if other programs are also taking up VRAM on the card. But in any case, the point is that my textures *would* be larger than normal for 800x600, but otherwise, they're optimized to not need mipmaps, if the user is running on a larger display.
Alex wrote:
Quote:It is possible to disable mipmaps to reduce VRAM usage, but you will undoubtedly increase the number of texture cache misses which will hurt performance far more than any VRAM savings will help.
Alex, could you elaborate a little bit on that? What you mean by texture cache misses? From my limited understanding, I'd guess that it would be a fillrate problem, not a texture miss problem... that is, using a texture optimized for 1680x1050 in a 800x600 window, would require a certain amount of scaling down on-the-fly without any mipmaps... I'm not sure why there would be texture cache misses? (which I would understand to mean textures not being in VRAM, and therefore needing to be loaded from main memory to VRAM before the object can be drawn) It seems like I'd have fewer of those, since without the mipmaps taking up VRAM, I could have all of my objects in VRAM, versus most of them, hopefully eliminating misses entirely. Fillrate would suffer on 800x600 somewhat, but if the game can run full-speed on 1680x1050, I'd think the fillrate should be about the same if something is scaled down to a lower res, although that's just a guess. (i.e. it's *reading* the same number of pixels per frame as it would on 1680x1050, but writing fewer, and since it's done in hardware, I'd guess the speed would be about the same for both scenarios... i.e. reading 1000 pixels and writing 1000 should be similar to reading 1000 and writing only 500. Again I'm guessing though.) Although if that's such a problem, maybe I'd add some code to the engine to scale down my textures on-the-fly when I first load them, once, if a lower-than-intended resolution is used by the game.
If I'm still being stupid, and really should keep mipmaps, let me know. But otherwise, if a game's textures are intelligently optimized for a specific resolution, and almost never scaled smaller than their original size (or if scaled smaller, it's only small amounts, like 70% to 90% -- something that would never result in a smaller mipmap being used anyway), then could it be beneficial to remove the mipmaps? Or am I still not understanding something?
My impression is that the fillrate problems without mipmaps might be overexaggerated? Would drawing a 256x256 texture to a 16x16 area really take any *longer* than drawing it to a 256x256 area? If not, I don't think it's a problem for my specific scenario. For most users, yes it would be, but for me no.
-Vern
#7
It's also worth noting that without mipmaps any textures that are reduced in size will have a very aliased look (mipmaps were created to solve this issue).
04/01/2008 (2:49 am)
When a texture is sampled a block of the texture is read into a texture cache, which can be read from much faster than the texture can be read from VRAM. If there is a large discontinuity between texture samples, as there would be when rendering a 256x256 texture to a 16x16 area, then the majority of samples cannot be read from the texture cache, thus you have a texture cache miss and start hammering VRAM, which significantly reduces performance. When mipmaps are available to use the card will use one of the mip levels instead of the full 256x256 texture, thus reducing cache misses.It's also worth noting that without mipmaps any textures that are reduced in size will have a very aliased look (mipmaps were created to solve this issue).
#8
Some cases where I have disabled/biased mip levels:
There are 'zodiac' images on the ground which circle players. The texture is 256x256 and is mostly has alpha values of 0. Once it mips down a few levels, the edges of the texture have gotten blurred so that where there was previously alpha values of 0, there is now color data. Since I am using clamp, for the texture addressing, there are a few pixels which get 'stretched', making it look like there are wings on the circle. To fix this, I checked the mip chain that was getting generated, with PIX, and decided that the lowest 3-4 mip levels should be ignored. So I put in a mip map LOD bias of -3, and that ensured that the smallest 3 mip levels would never get used. (This texture is generated at run-time, so the next solution doesn't apply)
Another way to do things, in TGEA, is to use .DDS image files with mip-maps already generated. You can control how many mip levels get stored in the file, so this is also a possible solution.
04/01/2008 (9:57 am)
Texture filtering can also pull from multiple mip-map levels. There are cases where it is useful to disable mip-mapping, or provide a mip-level bias to the API, but as a general rule, it is not a good idea. Some cases where I have disabled/biased mip levels:
There are 'zodiac' images on the ground which circle players. The texture is 256x256 and is mostly has alpha values of 0. Once it mips down a few levels, the edges of the texture have gotten blurred so that where there was previously alpha values of 0, there is now color data. Since I am using clamp, for the texture addressing, there are a few pixels which get 'stretched', making it look like there are wings on the circle. To fix this, I checked the mip chain that was getting generated, with PIX, and decided that the lowest 3-4 mip levels should be ignored. So I put in a mip map LOD bias of -3, and that ensured that the smallest 3 mip levels would never get used. (This texture is generated at run-time, so the next solution doesn't apply)
Another way to do things, in TGEA, is to use .DDS image files with mip-maps already generated. You can control how many mip levels get stored in the file, so this is also a possible solution.
#9
Thanks so much all, very helpful explanations all around!
04/01/2008 (11:11 am)
Pat, is it fairly simple to disable mipmaps in TGB? I could always test the results and see how it performs with them disabled, given that I don't think my game even uses them when in large resolution. But I'm not even sure where to look in the source code, or what to do once I find the right section(s). If it's too complicated for a semi-experienced C coder to tackle, that's fine, but if it's not terribly difficult, or you know of a document or thread that even hint at what to do (or you could explain it yourself), I'd be interested in hearing more. Otherwise, it's just good to know what I've learned already.Thanks so much all, very helpful explanations all around!
#10
04/01/2008 (11:32 am)
I have not done this in TGB. You are probably correct that a lot of TGB sprites do not need mip levels. Alex will be able to answer this better, but you should be able to do this by only passing 1 mip map level to OpenGL.
#11
and GL_NEAREST_MIPMAP_ to GL_LINEAR and GL_NEAREST, otherwise the card will attempt to read from mipmap levels which do not exist.
04/01/2008 (12:15 pm)
Textures are loaded in TextureManager::createGLName, which can be found at line 953 of gTexManager.cc. You can suppress mipmap generation by forcing the type of all textures to be BitmapTexture, or you can go through the createGLName method and replace for loops with a single call to glTex(Sub)Image2D. Also be sure to change all instances of GL_LINEAR_MIPMAP_
#12
04/01/2008 (1:29 pm)
Awesome, thanks Alex! 
Torque Owner Kevin James
Your only lucky break would be if Melv dropped in, or some GG employee who has worked on TGB. (or the other 10% of TGB developers who really know their stuff!)