Improving the terrain texture blending
by Lukas Joergensen · in Torque 3D Professional · 07/20/2013 (12:10 pm) · 63 replies
Hey guys, I looked into this myself but it seems the terrain texture uses procedural shaders for blending textures and thats wayy too complex for me atm so I thought I would post it here and hope that someone have a better understanding of the procedural shader system and might be able to implement this.
I read this article on Gamasutra and found it very interesting.
It goes through a set of algorithms to go from this kind of blending which is the one it seems T3D has:
To this much more accurate and natural blending:
(All images is from the article linked above)
Using this algorithm that seems simple enough:
This seems like something that could improve the terrain detail of T3D alot, so whats your thoughts? Would this be possible to do? And it would be awesome if anyone had a good enough understanding of the shader system to actually implement this.
Lastly what do you think about having to have a depth map? obviously it would be best if that could be omitted if desired, using a default depthmap or just a grayscale image.
I read this article on Gamasutra and found it very interesting.
It goes through a set of algorithms to go from this kind of blending which is the one it seems T3D has:
To this much more accurate and natural blending:
(All images is from the article linked above)
Using this algorithm that seems simple enough:
float3 blend(float4 texture1, float a1, float4 texture2, float a2)
{
float depth = 0.2;
float ma = max(texture1.a + a1, texture2.a + a2) - depth;
float b1 = max(texture1.a + a1 - ma, 0);
float b2 = max(texture2.a + a2 - ma, 0);
return (texture1.rgb * b1 + texture2.rgb * b2) / (b1 + b2);
}Where a1 and a2 is the opacity of the respectively texture1 and texture2 and the alpha channel of texture1 and texture2 is a depth map.This seems like something that could improve the terrain detail of T3D alot, so whats your thoughts? Would this be possible to do? And it would be awesome if anyone had a good enough understanding of the shader system to actually implement this.
Lastly what do you think about having to have a depth map? obviously it would be best if that could be omitted if desired, using a default depthmap or just a grayscale image.
About the author
IPS Bundle available at: https://www.winterleafentertainment.com/Products/IPS.aspx
#2
07/20/2013 (10:33 pm)
If you didn't want to use a depth map, cases like the one you demonstrate could be handled by using the greyscale value of the texture as the depth. That's what I thought was going on when I first looked at the comparison...
#3
Some time ago (2009) there was a discussion about specular maps for T3D terrain. At that time Tom Spillman pointed out that adding additional maps was too costly; the reason they weren't implemented. Of course things have improved looking at hardware, but the textures have grown in size as well over the years to meet up with the demands. So if you come to the point of adding a 4th map for T3D's terrain; what would you want to implement, a specular map or depth map?
07/20/2013 (11:05 pm)
Good find! T3D's terrain could use some improvement.Some time ago (2009) there was a discussion about specular maps for T3D terrain. At that time Tom Spillman pointed out that adding additional maps was too costly; the reason they weren't implemented. Of course things have improved looking at hardware, but the textures have grown in size as well over the years to meet up with the demands. So if you come to the point of adding a 4th map for T3D's terrain; what would you want to implement, a specular map or depth map?
#4
I would also like specular for terrain maps, just in case you need this function, does not mean it has to be on by default.
We already have a lot of maps for one terrain at them moment: base, macro, detail, normal and depthmap but this is integrated into the normal map and makes the parallax effect and if you have parallax blending textures, this looks different than the above image.
Here is an image how parallax textures blend, I did not have such big stones ready, but something similar:
I think the parallax covers the transition area a bit.
07/21/2013 (1:33 am)
Funny, I stepped over the exact same article yesterday...I would also like specular for terrain maps, just in case you need this function, does not mean it has to be on by default.
We already have a lot of maps for one terrain at them moment: base, macro, detail, normal and depthmap but this is integrated into the normal map and makes the parallax effect and if you have parallax blending textures, this looks different than the above image.
Here is an image how parallax textures blend, I did not have such big stones ready, but something similar:
I think the parallax covers the transition area a bit.
#5
@Nils this wouldn't actually be another map, but rather another channel for the diffuse map so it wouldn't be as hard on perfomance as a new map is.
@duion agreed if we are able to turn it off, the perfomance argument becomes a moot point.
07/21/2013 (3:23 am)
@dB in the article he is just using they grayscale image of the image so that can work (although I'd imagine you'd want to be able to set a custom one if the grayscale doesn't cut it.)@Nils this wouldn't actually be another map, but rather another channel for the diffuse map so it wouldn't be as hard on perfomance as a new map is.
@duion agreed if we are able to turn it off, the perfomance argument becomes a moot point.
#6
07/21/2013 (8:30 pm)
As far as "which one should we use" I like an approach that was proposed a while ago that would allow you to select one for each material - either use parallax, specular, or this new depth map.
#7
If an extra file load would be acceptable, you could think about a RGB file to store specular, parallax and depth in each channel. This way you can choose to have a different size and different compression then the existing ones. A downside is that these kind of textures would be T3D only.
07/21/2013 (10:40 pm)
Most textures I create are in the DDS format. For terrain textures I mostly use the DXT1 compression, even for normal maps if the distortion is acceptable. Everything is mipmapped and often do the terrain normal maps also have 2 or 3 mipmaps (with a slight blur) to reduce artefacts. I noticed when adding an alpha channel, I need to drop the DXT1 and end up with huge files. A good thing with DDS is that the file size gives an indication to what it does with memory load. I'm wondering if I would add an additional RGB-noAlpha texture it would be almost the same as adding an channel to an existing one (in my case).If an extra file load would be acceptable, you could think about a RGB file to store specular, parallax and depth in each channel. This way you can choose to have a different size and different compression then the existing ones. A downside is that these kind of textures would be T3D only.
#8
Konrads material accumulation prettymuch comes from edge detection on the normal maps if i recall correctly.
Comparing the two textures normal maps you could see which has the highest average and lowest average per pixel which would determine which texture gets the stronger blend between the two rather than the average of the two textures we get with the current blending.
07/22/2013 (10:12 am)
well rather than adding in another texture to read for the height, it should be possible to do something from the normal maps that could achieve something similar.Konrads material accumulation prettymuch comes from edge detection on the normal maps if i recall correctly.
Comparing the two textures normal maps you could see which has the highest average and lowest average per pixel which would determine which texture gets the stronger blend between the two rather than the average of the two textures we get with the current blending.
#9
Only in some cases when the pattern is very different they do not blend very good, like the example picture, but such big patterns do not happen very often.
I showed it in my picture, the grass blends well with the soil, but not with the very different pavement stones.
I would not add an extra map for such little advantage.
07/22/2013 (1:05 pm)
You already have a heightmap in the normal map alpha channel for parallax and for my taste they already blend very good.Only in some cases when the pattern is very different they do not blend very good, like the example picture, but such big patterns do not happen very often.
I showed it in my picture, the grass blends well with the soil, but not with the very different pavement stones.
I would not add an extra map for such little advantage.
#10
07/22/2013 (1:36 pm)
You will only have the height information in the alpha channel if you are using parallax, and to be fair, alot dont, as usually a normal map will do enough for them, so seeing if theres the possibility to use the normal maps basic colour information to compare the two is a lower requirement than making people add the height data into the alpha as a required texture, be it for parallax or blending.
#11
But I wonder if it would be possible to just use the brightness of the diffuse map to determine what blends and what not, just use the bright spots from one texture, to fill the dark spots of the other and the other way around, a heightmap basically is nothing else.
07/22/2013 (1:50 pm)
It is also there, if I turn it off ;) because I put a heightmap in almost every terrain normal map now.But I wonder if it would be possible to just use the brightness of the diffuse map to determine what blends and what not, just use the bright spots from one texture, to fill the dark spots of the other and the other way around, a heightmap basically is nothing else.
#12
Getting a solution into the public release of T3D to meet your own needs and requirements isnt what a community project is about, finding what the community requires and adapting the engine to meet their needs is what the position is about, knowing what people need, what we use and what we would like to see, being blinkered and taking the "my way is the right way" approach without any discussion or flexibility will not get anyone anywhere.
Why do so many people ask how parallax is done on terrains, if everyone already puts the height data into the normal maps alpha ? If they know to put it there, then they would surely know what they are putting it there for in the first place no ? ...
The basic texture maps are (not limited to terrains here, im including shapes, and im sure youre aware, or are you, that there isnt really any set correlation between standard use of each channel of a texture file when being used for terrains or shapes)
Diffuse
Normal
The next step people go onto use:
Detail
Specular
After that, people look at parallax for extra detail, and still often dont know where to start, where to put the height image, as there isnt an option to use a separate file for it like you can with the specular on a shape
07/22/2013 (2:31 pm)
Yes but Duion, presuming everyone does something just because you do it is a major folly and reeks of arrogance.Getting a solution into the public release of T3D to meet your own needs and requirements isnt what a community project is about, finding what the community requires and adapting the engine to meet their needs is what the position is about, knowing what people need, what we use and what we would like to see, being blinkered and taking the "my way is the right way" approach without any discussion or flexibility will not get anyone anywhere.
Why do so many people ask how parallax is done on terrains, if everyone already puts the height data into the normal maps alpha ? If they know to put it there, then they would surely know what they are putting it there for in the first place no ? ...
The basic texture maps are (not limited to terrains here, im including shapes, and im sure youre aware, or are you, that there isnt really any set correlation between standard use of each channel of a texture file when being used for terrains or shapes)
Diffuse
Normal
The next step people go onto use:
Detail
Specular
After that, people look at parallax for extra detail, and still often dont know where to start, where to put the height image, as there isnt an option to use a separate file for it like you can with the specular on a shape
#13
If this would be the normal map of the example image:
Used nDo2 to create this example: convert diffuse to normal
It can be converted to height with the right algorithm:
Used nDo2 to create this example: convert normal to displace
That would be better then using the RGB values of the diffuse map.
For example if you want to create something like this:
Then the rocks (low RGB values) would be covered with sand (high RGB values) while it should be the other way around. A normal map could be the right data for the job if you'd choose not to implement it in an extra channel.
The example height map looks almost good enough for parallax ;)
07/22/2013 (11:51 pm)
Yes of course, @Andy you're right! We already have some height data in the normal map!If this would be the normal map of the example image:
Used nDo2 to create this example: convert diffuse to normal
It can be converted to height with the right algorithm:
Used nDo2 to create this example: convert normal to displace
That would be better then using the RGB values of the diffuse map.
For example if you want to create something like this:
Then the rocks (low RGB values) would be covered with sand (high RGB values) while it should be the other way around. A normal map could be the right data for the job if you'd choose not to implement it in an extra channel.
The example height map looks almost good enough for parallax ;)
#14
Creating heightmaps for parallax is a bit tricky because you need to blur it to reduce artifacts in parallax.
Ron also suggested njob as a free alternative and it is pretty good, I use it all the time now, you can also automatically can generate heightmaps out of normal maps there.
07/23/2013 (1:43 am)
Yes it can be hard to figure out how to make heightmap and put it into the normal map, but Ron made a tutorial for it with gimp 3tdstudios.com/tutorials.html this helped me a lot and for photoshop there are already tutorials.Creating heightmaps for parallax is a bit tricky because you need to blur it to reduce artifacts in parallax.
Ron also suggested njob as a free alternative and it is pretty good, I use it all the time now, you can also automatically can generate heightmaps out of normal maps there.
#15
but I did find this interesting document (pdf).
It shows a "simple" algorithm:
I'm a designer, it's pretty useless in my hands, too bad :)
07/23/2013 (3:04 am)
There isn't much (useful) information about turning normal maps into to height maps, but I did find this interesting document (pdf).
It shows a "simple" algorithm:
I'm a designer, it's pretty useless in my hands, too bad :)
#17
It's very simple: Roughly spoken uses a 8bit RGB texture with a 4th channel (for alpha, parallax, whatever...) 100% more memory then a 4bit texture with 3 channels (RGB). If you create a game, every byte you put in is a byte to many. You should always save whenever possible. Optimization is utterly important while developing a game. The very same reason you should use DDS instead of PNG or JPG or whatever.
It doesn't mean I want to use the normal map; we should look at the possibility. If converting a normal map to a height map uses more recourses then other methods; you of course don't go for that.
Edit: And if this method of converting normal maps to height maps also turns out to be good for adding a parallax effect without creating the channel by hand... perhaps... :)
07/23/2013 (5:54 am)
@Duion; I thought it was mentioned before, but maybe a bit unclear. It's very simple: Roughly spoken uses a 8bit RGB texture with a 4th channel (for alpha, parallax, whatever...) 100% more memory then a 4bit texture with 3 channels (RGB). If you create a game, every byte you put in is a byte to many. You should always save whenever possible. Optimization is utterly important while developing a game. The very same reason you should use DDS instead of PNG or JPG or whatever.
It doesn't mean I want to use the normal map; we should look at the possibility. If converting a normal map to a height map uses more recourses then other methods; you of course don't go for that.
Edit: And if this method of converting normal maps to height maps also turns out to be good for adding a parallax effect without creating the channel by hand... perhaps... :)
#18
So I prefer to press ctrl+c and ctrl+v to copy a heightmap into the texture, it is not that hard.
07/23/2013 (6:17 am)
It may sound like a great idea to have it all autocalculated out of the normal map, but likely it will turn out ugly and buggy, because you have no control over how the pattern turns out and no control to remove flickering artifacts.So I prefer to press ctrl+c and ctrl+v to copy a heightmap into the texture, it is not that hard.
#19
If thats the scenario, why does any programmer ever try everything. I'm pretty sure the guys that originally came up with procedural generation of things such as terrains, textures, even models all had people going "why bother" but if they never stayed with that idea and developed it, then we wouldnt have advanced technologies now would we ?
Imagine the guy that said "why bother with a computer, we have typewriters"...
If using the normal to generate this form of blending, aswell as possibly have a cache of the generated height data for use for parallax would that not be two cookies for the price of one at half the file memory usage ?
And also Duion, its not just ctrl+c and ctrl+v, youre also running your texture through another program either for filters or effects to get the final result of your heightmap, and im pretty sure that a PC is going to be able to automate that faster on the fly that youre doing it yourself in your external program. Yes, hand tweaking each parallax to suit its texture is something i do myself, take a moment and look at my previous terrain threads if you want, im not an idiot, and yes, i know about Ron's tutorial, you may want to check with Ron where he got those methods from.
Before you try to explain to me what a parallax map is lol... heres a 3 or 4 year old screenshot
Also were not using the normal map to find the blending, we are making a procedurally generated heightmap (displacemenmt) from the normal map on the fly, automating it so that is a feature that can be accessed by all who have just a basic set of diffuse and normal map, not just those who are in the know of how to make a texture and where to add the displacement data for the parallax.
This could even be a "check" where if a normal map has the alpha channel there with data, it will use that, where as if it does not, it will make an automated version for you.
What would be the downside to that at the end of the day ? And seriously dont tell me "i can just make it externally and copy and paste it into the alpha to achieve the same thing" ...
07/23/2013 (2:22 pm)
And again...If thats the scenario, why does any programmer ever try everything. I'm pretty sure the guys that originally came up with procedural generation of things such as terrains, textures, even models all had people going "why bother" but if they never stayed with that idea and developed it, then we wouldnt have advanced technologies now would we ?
Imagine the guy that said "why bother with a computer, we have typewriters"...
If using the normal to generate this form of blending, aswell as possibly have a cache of the generated height data for use for parallax would that not be two cookies for the price of one at half the file memory usage ?
And also Duion, its not just ctrl+c and ctrl+v, youre also running your texture through another program either for filters or effects to get the final result of your heightmap, and im pretty sure that a PC is going to be able to automate that faster on the fly that youre doing it yourself in your external program. Yes, hand tweaking each parallax to suit its texture is something i do myself, take a moment and look at my previous terrain threads if you want, im not an idiot, and yes, i know about Ron's tutorial, you may want to check with Ron where he got those methods from.
Before you try to explain to me what a parallax map is lol... heres a 3 or 4 year old screenshot
Also were not using the normal map to find the blending, we are making a procedurally generated heightmap (displacemenmt) from the normal map on the fly, automating it so that is a feature that can be accessed by all who have just a basic set of diffuse and normal map, not just those who are in the know of how to make a texture and where to add the displacement data for the parallax.
This could even be a "check" where if a normal map has the alpha channel there with data, it will use that, where as if it does not, it will make an automated version for you.
What would be the downside to that at the end of the day ? And seriously dont tell me "i can just make it externally and copy and paste it into the alpha to achieve the same thing" ...
#20
07/24/2013 (7:13 pm)
I truly respect @Andy's patience and whish I had some of that, but @Duion; try to be a bit open to other one's ideas and suggestions and focus somehow on the subject instead of your own ideas, your own methods and your own aesthetics of something that's not even an issue. 
Torque Owner Lukas Joergensen
WinterLeaf Entertainment