Normal map and DDS
by Steve Acaster · in Artist Corner · 04/18/2009 (1:57 pm) · 16 replies
Whilst it gives excellent compression, DDS format has a habit of mangling normal maps. Many will come out with a clear pixel effect in places. Some type of images seem better suited to .dds than others, but even then there is a plethora of normal and .dds options.
Gimp
DXT5 with color selection set to Inset Bounding Box seems to give the closest result to the jpg. However, whilst a heavily pitted surface finally exported well, a "scraped concrete texture" retains a standardized pixel distortion.
Photoshop
The Nvidia Photoshop plug-ins for both normal mapping and dds format come with a damn sight more options than documentation for those options.
Utilizing the preview option becomes essentail. Even so, as in the Gimp tests, the "scraped concrete texture" fairs badly in-game, even though apparently good on preview and export.
After lingering around google on the subject I found that having the Normal Map Conversion in the DDS exporter set to "convert to tangent space normal" helped to ease some pixelation problems, at the expense of dramatically smoothing and reducing the exported normal map.
Tangent space is heavily optioned in the "Image Options" of the dds exporter, but appears to have no documentation on what the many options actually do, or abbreviation mean. (RMS anyone?) Object Space and Greyscale both seem to cause heavy pixel distortion.
Color Map can disable the pixelation, but is much darker, and does not show correct specular lighting.
"scraped concrete texture"
To be honest I've noticed that the heavy pixelation areas only really show (once as successfully exported as possible) from one side, however it is a little much to ask that the player walks down an environment in only one direction and never turns around.
I'm led to believe that "pink" normal maps will only work on Nvidia cardsets, and even then they cause many problems.
If anyone has any useful tricks, tips or know-how on Normal map compression for DDS format, feel free to comment below.
Gimp
DXT5 with color selection set to Inset Bounding Box seems to give the closest result to the jpg. However, whilst a heavily pitted surface finally exported well, a "scraped concrete texture" retains a standardized pixel distortion.
Photoshop
The Nvidia Photoshop plug-ins for both normal mapping and dds format come with a damn sight more options than documentation for those options.
Utilizing the preview option becomes essentail. Even so, as in the Gimp tests, the "scraped concrete texture" fairs badly in-game, even though apparently good on preview and export.
After lingering around google on the subject I found that having the Normal Map Conversion in the DDS exporter set to "convert to tangent space normal" helped to ease some pixelation problems, at the expense of dramatically smoothing and reducing the exported normal map.
Tangent space is heavily optioned in the "Image Options" of the dds exporter, but appears to have no documentation on what the many options actually do, or abbreviation mean. (RMS anyone?) Object Space and Greyscale both seem to cause heavy pixel distortion.
Color Map can disable the pixelation, but is much darker, and does not show correct specular lighting.
"scraped concrete texture"
To be honest I've noticed that the heavy pixelation areas only really show (once as successfully exported as possible) from one side, however it is a little much to ask that the player walks down an environment in only one direction and never turns around.
I'm led to believe that "pink" normal maps will only work on Nvidia cardsets, and even then they cause many problems.
If anyone has any useful tricks, tips or know-how on Normal map compression for DDS format, feel free to comment below.
About the author
One Bloke ... In His Bedroom ... Making Indie Games ...
#2
Also if you really want to use DTX compression for Normals and are not using an alpha channel (ie. TGEA) you can use DTX1 instead. The primary difference is that DTX1 has only a 2 bit alpha channel, while DTX3 and 5 have an 8 bit alpha.
With regards to your normal map generation, for the most part you will always want to utilize Tangent space. Here is a really good bit of documentation on the different type of normal spaces you can generate from the 3DS Max docs:
One other helpful though is to ensure that you use a 128:128:255 color for your background for your normal map (and not black or any other color). This will help prevent any artifacts from creeping into your model as the mip map compression takes effect, 128:128:255 is the color for 'no change' in a normal map and anything else will do subtle changes to the normal map result.
04/18/2009 (2:16 pm)
One other thought is to actually use the 8:8:8 or 8:8:8:8 compression method instead of DTX5 for your DDSs.Also if you really want to use DTX compression for Normals and are not using an alpha channel (ie. TGEA) you can use DTX1 instead. The primary difference is that DTX1 has only a 2 bit alpha channel, while DTX3 and 5 have an 8 bit alpha.
With regards to your normal map generation, for the most part you will always want to utilize Tangent space. Here is a really good bit of documentation on the different type of normal spaces you can generate from the 3DS Max docs:
Quote:There are four methods for projecting the normals:
WorldProject using world coordinates. This is useful mainly for objects that don't move or deform; otherwise, a moving object with world-projected normals will appear to “swim” through the texture.
ScreenProject using screen coordinates; that is, flat projection in the Z axis. This method is useful mainly for stationary objects seen from a single angle only; for example, a statue seen through a window.
Local XYZProject using the object's local coordinates. This method can be used for stationary or moving objects, but not for objects that deform: if the object deforms, the projection will appear incorrect at some frames.
Tangent(The default.) Project at a tangent to the target object's surface. This is the method to use for objects that both move and deform, such as animated characters.
One other helpful though is to ensure that you use a 128:128:255 color for your background for your normal map (and not black or any other color). This will help prevent any artifacts from creeping into your model as the mip map compression takes effect, 128:128:255 is the color for 'no change' in a normal map and anything else will do subtle changes to the normal map result.
#3
Do not use the nVidia Photoshop plugin to generate normal maps. Even the guys at nVidia will tell you its crap and really doesnt generate information right!
Instead you should look at things like Crazybump, PixPlant or ShaderMap Pro if you want to generate normal maps from a 2d image.
However it should be noted that the absolute best normal maps will only be generated from referencing 3D data (low against high).
04/18/2009 (2:24 pm)
Oh and one other thing.Do not use the nVidia Photoshop plugin to generate normal maps. Even the guys at nVidia will tell you its crap and really doesnt generate information right!
Instead you should look at things like Crazybump, PixPlant or ShaderMap Pro if you want to generate normal maps from a 2d image.
However it should be noted that the absolute best normal maps will only be generated from referencing 3D data (low against high).
#4
It gives a clean texture/map without the "artifacts" discussed here when using "color selection" = "inset bounding box" and mipmaps set to "Lanczos".
04/25/2009 (4:10 pm)
After much playing with various toolsets, most of them open source, some commercial (not including CrazyBump which does seem a popular choice - I didn't want to pay the higher commercial license fee) I found that GIMP does have the better compression for a DXT5 normal map (made from a texture and not 3D data).It gives a clean texture/map without the "artifacts" discussed here when using "color selection" = "inset bounding box" and mipmaps set to "Lanczos".
#5
save yourself bandwidth since I would like to see this at decent res(if you choose post an image or five)
if post_images = true
{
thx
}
else
{
k..np
}
04/25/2009 (6:33 pm)
..mind posting a few images Steve ?save yourself bandwidth since I would like to see this at decent res(if you choose post an image or five)
if post_images = true
{
thx
}
else
{
k..np
}
#6
Okay, admittedly my new textures aren't the same as my old ones, I'm in the process of redoing them. Currently they are 512px photos save for my player and weapon artwork which are 512px custom textures.
torque.abigholeintheweb.com/public_system/useruploads/Gimp512_1.jpg
torque.abigholeintheweb.com/public_system/useruploads/GIMP512_2.jpg
torque.abigholeintheweb.com/public_system/useruploads/GIMP512_3.jpg
torque.abigholeintheweb.com/public_system/useruploads/GIMP512_4.jpg
And the orginal error/artifact problem on older textures - pixelation.
torque.abigholeintheweb.com/public_system/useruploads/PS7x1024_prob1.jpg
torque.abigholeintheweb.com/public_system/useruploads/PS7x1024_prob2.jpg
caution big 1680x1050s
04/25/2009 (6:56 pm)
Sure, hang on...Okay, admittedly my new textures aren't the same as my old ones, I'm in the process of redoing them. Currently they are 512px photos save for my player and weapon artwork which are 512px custom textures.
torque.abigholeintheweb.com/public_system/useruploads/Gimp512_1.jpg
torque.abigholeintheweb.com/public_system/useruploads/GIMP512_2.jpg
torque.abigholeintheweb.com/public_system/useruploads/GIMP512_3.jpg
torque.abigholeintheweb.com/public_system/useruploads/GIMP512_4.jpg
And the orginal error/artifact problem on older textures - pixelation.
torque.abigholeintheweb.com/public_system/useruploads/PS7x1024_prob1.jpg
torque.abigholeintheweb.com/public_system/useruploads/PS7x1024_prob2.jpg
caution big 1680x1050s
#7
torque.abigholeintheweb.com/public_system/useruploads/GIMPx1024_1.jpg
There's some very minor pixelation but it's much more subdued and harder to spot. I've found that compressing normal maps with high "noise" type effects create the most problems.
04/25/2009 (7:59 pm)
And after a hunt for my old texture source files, here's a GIMPed version of PS7x1024_prob1 for better/easier comparison.torque.abigholeintheweb.com/public_system/useruploads/GIMPx1024_1.jpg
There's some very minor pixelation but it's much more subdued and harder to spot. I've found that compressing normal maps with high "noise" type effects create the most problems.
#9
There's still some artifacting/pixelation on the mipmaps of the "noise map" (last image link I posted) which appears the hardest thing for the DDS to compress cleanly. Custom mipmaps are still on the "to-do" list of the plugin creator.
I also found that there were new version of GIMP, and both the DDS and Normal Map plugins available.
GIMP
GIMP DDS PLugin
GIMP Normal Map Plugin
And while I'm here spamming links ... I was removing dead links from my Bookmarks in a little Spring cleaning and then searching for newer versions and found:
The Starving Developers' QuickStart Guide '08
A big, long list of free or nearly free applications for 2D, 3D, reference and general game developing links.
04/27/2009 (2:54 am)
After even more playing around I found that changing "Inset Bounding Box" to "Distance" helped sort out a slight artifact problem with the edges of a circlular rock/brick texture I had.There's still some artifacting/pixelation on the mipmaps of the "noise map" (last image link I posted) which appears the hardest thing for the DDS to compress cleanly. Custom mipmaps are still on the "to-do" list of the plugin creator.
I also found that there were new version of GIMP, and both the DDS and Normal Map plugins available.
GIMP
GIMP DDS PLugin
GIMP Normal Map Plugin
And while I'm here spamming links ... I was removing dead links from my Bookmarks in a little Spring cleaning and then searching for newer versions and found:
The Starving Developers' QuickStart Guide '08
A big, long list of free or nearly free applications for 2D, 3D, reference and general game developing links.
#10
I'm liking that Starving Developers' QuickStart Guide -- lots of great tools on there.
04/27/2009 (4:28 am)
Ah cool. I knew about the updated Gimp and NormalMap plug-in, but missed the DDS plug-in, thanks for spamming.I'm liking that Starving Developers' QuickStart Guide -- lots of great tools on there.
#11
And I did patch all the reported errors in the engine. Reverting to non-DDS formats for normal mapping cures said problem. This wasn't an issue in 1.7.1.
edit: Submitted as a bug report.
04/29/2009 (8:08 am)
After a fair bit of porting/testing/code patching, it appears that TGEA 1.8.1 does not support Normal Map DDS files on DTS models. It all goes horribly wrong with inverted lighting.And I did patch all the reported errors in the engine. Reverting to non-DDS formats for normal mapping cures said problem. This wasn't an issue in 1.7.1.
edit: Submitted as a bug report.
#13
Did anyone find any dds compressor conversion tools that work well with 3d normal map output? Both the tools above (gimp and dds converter 2) has worse compressed normal map conversion output than nvidias photoshop with the tangent space on. Especially in red of course.
Since the developer is using an older TGEA, we don't have dxt-nm. Which means currently I'm using dxt5, and crying everytime I see the results of smoothed high spec mechanical imagery in the engine. Especially after all the work put into the high rez versions. So other than having a file size 3X bigger with 8.8.8.8. Any other options? (We are using alpha for spec - long story).
(Steve you should use xNormals normal map ps plugin.
http://www.xnormal.net/1.aspx
Soo much nicer results. Plus you can create AOs as well.)
The thing is, xnormals built in previewer engine shows less pronounced artifacting than in the TGEA engine. Who knows, maybe its da lighting.
09/16/2009 (4:32 am)
Hey this is only what.. 5-6? months old. :DDid anyone find any dds compressor conversion tools that work well with 3d normal map output? Both the tools above (gimp and dds converter 2) has worse compressed normal map conversion output than nvidias photoshop with the tangent space on. Especially in red of course.
Since the developer is using an older TGEA, we don't have dxt-nm. Which means currently I'm using dxt5, and crying everytime I see the results of smoothed high spec mechanical imagery in the engine. Especially after all the work put into the high rez versions. So other than having a file size 3X bigger with 8.8.8.8. Any other options? (We are using alpha for spec - long story).
(Steve you should use xNormals normal map ps plugin.
http://www.xnormal.net/1.aspx
Soo much nicer results. Plus you can create AOs as well.)
The thing is, xnormals built in previewer engine shows less pronounced artifacting than in the TGEA engine. Who knows, maybe its da lighting.
#14
Here is a package I zipped up a few months ago for someone: torque.abigholeintheweb.com/public_system/useruploads/shader_replacement.zip
***BE SURE TO BACKUP the shaders you're going to overwrite with that package***
As for better looking on compressed Normal Maps..good luck on that. :/
09/16/2009 (5:06 am)
..you may want to replace the bump shader in TGEA with a normal map shader. Not that this will fix your rendering results 100% but it will help out a bit.Here is a package I zipped up a few months ago for someone: torque.abigholeintheweb.com/public_system/useruploads/shader_replacement.zip
***BE SURE TO BACKUP the shaders you're going to overwrite with that package***
As for better looking on compressed Normal Maps..good luck on that. :/
#15
09/16/2009 (8:21 am)
On a side note, T3D has this whole issue of "artifacts_due_to_compression" resolved thanks to the compatibility of DX5_NM.
#16
That simply makes no sense. The compression (and thus size) of DDS files is fixed. All tools generate the same size for the same settings. For example, a 1024x1024 DDS with DXT1 and no MIP maps will be the same size no matter what tool generates it. It's not like PNG or JPG where the file size of a given texture size may vary.
It may be that NVIDIA generally generates MIP maps (which is a GOOD THING, because you don't want to re-generate/re-compress DXT MIP maps at runtime), and other tools don't. That will cause the files to be about 33% bigger than without MIP maps.
And, finally, yes, DXT5_NM is the bomb.
09/18/2009 (2:14 am)
Quote:generates larger DDSs in comparison to other apps
That simply makes no sense. The compression (and thus size) of DDS files is fixed. All tools generate the same size for the same settings. For example, a 1024x1024 DDS with DXT1 and no MIP maps will be the same size no matter what tool generates it. It's not like PNG or JPG where the file size of a given texture size may vary.
It may be that NVIDIA generally generates MIP maps (which is a GOOD THING, because you don't want to re-generate/re-compress DXT MIP maps at runtime), and other tools don't. That will cause the files to be about 33% bigger than without MIP maps.
And, finally, yes, DXT5_NM is the bomb.
Associate Logan Foster
perPixel Studios
What I recommend using instead is a free 3rd party app such as DDS Converter 2 (does not work on Vista). Its a pretty simple graphical tool which can do batch conversion of various files from one format to another and best of all it really optimizes the hell out of the DDS format without causing any extra visual problems.