Normals in Torque3D
by Duion · in Artist Corner · 12/09/2013 (1:35 pm) · 16 replies
Is there any documentation about the normal map orientations in Torque?
What I could find out so far is, that Torque needs normal maps with inverted y-channel to work properly.
But then I noticed that for terrain materials this is all different, it looks like that Torque there needs normal maps without the y-channel inverted, but even then, the x-channel is wrong and if you have heightmap in the alpha channel for parallax the x-channel stays wrong, but if you remove the heightmap, you can make it work properly by inverting the x-channel also.
Here is an example, these are simple bumps on a flat terrain, there is just one light in the whole scene, which is directly in front of the player.
First scene normal map with inverted y-channel as it is correct for shapes, as you can see it is all wrong, the light sides are dark and everything left is lit and everything right is dark, but it is a light that shines into all directions equally.
This is the same scene with y-channel back inverted, to a non inverted y-channel. The y-channel is correct, but the x-channel is still wrong, inverting does not help, if you use a heightmap for parallax in the alpha channel.
So is this a shader bug or is there something done wrong? I suppose the textures get mapped mirrored on the terrain, so everything gets wrong, a similar issue there is on sketch tool shapes, there the textures are mirrored also and the normals are all wrong.
What I could find out so far is, that Torque needs normal maps with inverted y-channel to work properly.
But then I noticed that for terrain materials this is all different, it looks like that Torque there needs normal maps without the y-channel inverted, but even then, the x-channel is wrong and if you have heightmap in the alpha channel for parallax the x-channel stays wrong, but if you remove the heightmap, you can make it work properly by inverting the x-channel also.
Here is an example, these are simple bumps on a flat terrain, there is just one light in the whole scene, which is directly in front of the player.
First scene normal map with inverted y-channel as it is correct for shapes, as you can see it is all wrong, the light sides are dark and everything left is lit and everything right is dark, but it is a light that shines into all directions equally.
This is the same scene with y-channel back inverted, to a non inverted y-channel. The y-channel is correct, but the x-channel is still wrong, inverting does not help, if you use a heightmap for parallax in the alpha channel.
So is this a shader bug or is there something done wrong? I suppose the textures get mapped mirrored on the terrain, so everything gets wrong, a similar issue there is on sketch tool shapes, there the textures are mirrored also and the normals are all wrong.
About the author
http://www.duion.com - Human Paradigm Specialist
#2
12/09/2013 (4:23 pm)
The bigger problem I have at the moment is how to make a standard when producing terrain materials, since the normal maps will be wrong. If you make a package for others, how you decide which standard you use? With heightmap or without, y-inverted or not, x-inverted or not there are many combinations. 
#3
12/10/2013 (7:37 am)
@Duion, you should add an issue to github with this information, not to be forgotten.
#4
Dxt3 is also the only way parallax textures work on normal surfaces like static shapes etc. Drawback is, that you don't have the smooth gradients, that you have in dxt5 compression.
But y-axis still needs to be the other way around on terrain than on normal materials (non-inverted y)
So maybe someone will find out, why only .dds files in dxt3 work and all others don't, even uncompressed formats are wrong.
12/14/2013 (5:42 pm)
Found a solution, just use dxt3.Dxt3 is also the only way parallax textures work on normal surfaces like static shapes etc. Drawback is, that you don't have the smooth gradients, that you have in dxt5 compression.
But y-axis still needs to be the other way around on terrain than on normal materials (non-inverted y)
So maybe someone will find out, why only .dds files in dxt3 work and all others don't, even uncompressed formats are wrong.
#5
12/15/2013 (5:01 pm)
Ok, dxt3 works very bad in most cases, at least I was happy for a short time.
#6
If you remove the alpha channel from a normal map with height in alphachannel and use this normal map, then you get the same problem as in the first picture, one side is bright and one is dark.
So the problem is that the additional heightmap somehow overrides the other normalmap channels and messes it up, but only in dxt5 while dxt3 works, maybe someone else gets an idea about this.
01/10/2014 (12:03 pm)
I came across the same problem today using a wrongly made normal map on a shape, which caused the same effect.If you remove the alpha channel from a normal map with height in alphachannel and use this normal map, then you get the same problem as in the first picture, one side is bright and one is dark.
So the problem is that the additional heightmap somehow overrides the other normalmap channels and messes it up, but only in dxt5 while dxt3 works, maybe someone else gets an idea about this.
#7
On another note i create a PR here github.com/GarageGames/Torque3D/pull/715 that is somewhat related to this, it allows parallax mapping when using dxt5nm on both regular materials and terrain materials.
07/03/2014 (1:06 am)
With the terrain when you use dxt5 it assumes you are using dxt5nm which is just a swizzling trick. I won't bother explaining it because there is a good explanation available here tech-artists.org/wiki/Normal_map_compressionOn another note i create a PR here github.com/GarageGames/Torque3D/pull/715 that is somewhat related to this, it allows parallax mapping when using dxt5nm on both regular materials and terrain materials.
#8
07/03/2014 (4:25 am)
I get an idea about it now, but I wonder if it will work properly, since on swizzling he blanks out 2 channels, where is the heightmap stored then? Why not tell him to not assume to use dxt5nm, tell him it is dxt3 or so instead.
#9
07/03/2014 (4:27 am)
It is not mandatory to blank out the two channels, it just gives better results if you do. So you can still chuck the height into the red channel. Either way it should still give better visual results than using dxt3
#10
07/03/2014 (4:33 am)
So I need to change the exporter for the .dds-normals also? I just meant to make him assume it is dxt3, when it is in reality dxt5, since dxt3 works fine. I have no idea about the code, but I think it cannot be that hard to make him stop using the dxt5nm thing and just using the red channel is not a good solution, since it is part of the normal map lighting, not a proper heightmap, or am I wrong?
#11
Which exporter are we talking about here?
07/03/2014 (4:38 am)
In the code whenever T3D detects a normal map using dxt5 is treats it as dxt5nm no matter what.Which exporter are we talking about here?
#12
07/03/2014 (4:43 am)
The NVIDIA dds exporter? It always blanks out two channels on dxt5nm, there is no more heightmap then.
#13
With photoshop (or gimp) you get your regular normal map,now move the red channel to alpha channel, fill the blue channel as a solid color and chuck you height into the red channel. Now save this as a regular dxt5 (not dxt5nm). There is no difference in the format between dxt5 and dxt5nm it's the same thing, it's just when you select dxt5nm the exporter will save you the hassle of manually moving the channels around. If you don't require parallax support for any particular normal map than by all means save ya self the hassle and just export as dxt5nm.
Hope that all makes sense, once you do it a few times it's not as complicated as it may sound ;-)
07/03/2014 (4:50 am)
You mean the add-in with photoshop?With photoshop (or gimp) you get your regular normal map,now move the red channel to alpha channel, fill the blue channel as a solid color and chuck you height into the red channel. Now save this as a regular dxt5 (not dxt5nm). There is no difference in the format between dxt5 and dxt5nm it's the same thing, it's just when you select dxt5nm the exporter will save you the hassle of manually moving the channels around. If you don't require parallax support for any particular normal map than by all means save ya self the hassle and just export as dxt5nm.
Hope that all makes sense, once you do it a few times it's not as complicated as it may sound ;-)
#14
By the way, why you use the red channel and why you can just move the red channel to alpha?
The parallax is an important thing for me on the terrain, since I want the possibility to use it when I want. If I want to save me the hassel, I will just use png, also an ugly workaround, but it works. Only remains the confusing y-axis inversion thing.
07/03/2014 (5:09 am)
OK, now I understand, but I am still not sure if I should use it, or just stick to png, I may try it out the next time I do terrain materials.By the way, why you use the red channel and why you can just move the red channel to alpha?
The parallax is an important thing for me on the terrain, since I want the possibility to use it when I want. If I want to save me the hassel, I will just use png, also an ugly workaround, but it works. Only remains the confusing y-axis inversion thing.
#15
Honestly if you are just playing around and it's not a production environment i wouldn't bother with it ;-). Any of the dxtN family have big benefits when it comes to saving VRAM, when you have a heap of textures it can make a massive difference. Having said that i do believe T3D ships with squish so i think an uncompressed texture such as PNG will be compressed (well it will be uncompressed from png format and than compressed again to a suitable compression format for the GPU to use). Haha don't quote me on that last one ;-)
I haven't noticed the inverted Y thing actually, i'll check it out.
07/03/2014 (5:25 am)
The red channel thing is only for dxt5 normal maps because currently T3D doesn't support parallax with dxt5. When you save a normal map using dxt5nm setting it is automatically moving the red channel to the alpha channel for you. The reason for this is because dxt5 uses the following size (R|G|B|A) 5|6|5|8 as you can see green and alpha have the highest bit depth so this produces better results. This dxt5 trick is tried and tested and has been used in plenty of AAA titles.Honestly if you are just playing around and it's not a production environment i wouldn't bother with it ;-). Any of the dxtN family have big benefits when it comes to saving VRAM, when you have a heap of textures it can make a massive difference. Having said that i do believe T3D ships with squish so i think an uncompressed texture such as PNG will be compressed (well it will be uncompressed from png format and than compressed again to a suitable compression format for the GPU to use). Haha don't quote me on that last one ;-)
I haven't noticed the inverted Y thing actually, i'll check it out.
#16
When I checked Ron's art packs for it, I think he knew about it, since his terrain textures were correct in Torque I think.
Especially producers of art packs should care about these issues, since their product may not be compatible with Torque3D or will look all wrong.
I know of the following issues:
1.Terrain normals need not inverted y, while other textures do.
2.DXT5 does not work for terrain textures with parallax.
3.Normal map seams on a model that has an normal map on UV layout.
4.Do not use the sketch tool to test your textures, like I did some time ago, since the convex shapes produced by the sketch tool have all wrong normals again, I think they are Y-inverted, but then Y and X are mirrored, so both are wrong.
5.Some problem of the tangent bases normals, which has been fixed I think.
07/03/2014 (10:42 am)
It is common practice to invert the Y-channel of the normal maps, since all the big engines handle it this way and it also looks visually more correct, Torque of course uses inverted Y-axis-normal maps, too, but not on terrain, on terrain it is the other way around, I wonder that nobody noticed.When I checked Ron's art packs for it, I think he knew about it, since his terrain textures were correct in Torque I think.
Especially producers of art packs should care about these issues, since their product may not be compatible with Torque3D or will look all wrong.
I know of the following issues:
1.Terrain normals need not inverted y, while other textures do.
2.DXT5 does not work for terrain textures with parallax.
3.Normal map seams on a model that has an normal map on UV layout.
4.Do not use the sketch tool to test your textures, like I did some time ago, since the convex shapes produced by the sketch tool have all wrong normals again, I think they are Y-inverted, but then Y and X are mirrored, so both are wrong.
5.Some problem of the tangent bases normals, which has been fixed I think.
Associate Ron Kapaun
3tdstudios.com
I have seen this as well, near as I can tell from glancing at the code, the terrain materials and the object materials are handled separately and differently. I 'think' there were two separate teams or coders for those features when the engine was built. This is part of the problem I am having in unifying the material tool to work across the board.
Ron