Improved texture blending in T3D!
by Lukas Joergensen · 06/25/2014 (3:08 pm) · 18 comments
Notice how the sand blends into the cracks in the rock.
I'll try to keep this one short and with lots of images to keep your attention!
So I've implemented an alternative terrain texture blending algorithm, based on this article by Andrew Mishkinis.
So apart from showing off my work I wanted to raise a discussion, do you want this in stock T3D? If not then I don't bother to clean it up etc. It should be noted that you can turn the blending on and off in real-time by just changing a boolean, so it doesn't really prevent you from using the linear interpolation for blending.
IF we want it in stock T3D then here are some points for discussion and research:
Current issues
It seems that there can be some hard edges, the reason for this has not yet been found.It seems like the surrounding textures affect the a texture far into where there should be no more blending. Seems to correlate with the lowest points of that texture.
We need to decide where to load the heightmap from, a popular idea is to calculate it from the normal map, but this means that normal maps are a must for this kind of blending (which seems like a reasonable demand). The parallax map in the alpha channel of the normal map can also be used since it's just a heightmap.
Current ideas
Per-texture scaling, right now all heightmaps are normalized, a small rock might have a height of 1 in a sand texture but an equivalent rock would have a height of 0.2 in a rocky texture. By letting the rock-texture have a higher "height-scale" then blending might improve.Thoughts? Discuss!
More images:
Before / After
Edit: It should be noted that the above example images are just there to show the effect and does not show the potential of the algorithm. A proper heightmap and better textures would improve the results a lot
About the author
IPS Bundle available at: https://www.winterleafentertainment.com/Products/IPS.aspx
#2
06/25/2014 (6:20 pm)
I'm liking what I'm seeing there. Keep up the great work Lukas! 
#3
06/25/2014 (7:17 pm)
I totally remember reading this article, awesome to see it in action! The results look much better imho than the faded outlines! 
#5
06/25/2014 (11:52 pm)
Very, very nice...
#6
I think I'll make it read the heightmap from the parallax map as it seems like it's most sensible, also as Duion points out, it only takes 5 minutes, and you can always set parallax scale to 0 if you don't want it.
Konrad Kiss also suggested using the normal map as a shallow, depth detail offset map, which I'll look a little more into. If any of you have some information on how to do this I'd be happy to learn more! Otherwise I'll just work my way from the discription Konrad gave me.
Anyways I'm glad you like the new blending, I was afraid that I would give you the wrong impression with my poor programmers-art heightmaps :P I'll keep you updated!
06/26/2014 (6:47 am)
@Duion I haven't made it blend parallax yet, that will be interesting to see. It could look both good and bad.I think I'll make it read the heightmap from the parallax map as it seems like it's most sensible, also as Duion points out, it only takes 5 minutes, and you can always set parallax scale to 0 if you don't want it.
Konrad Kiss also suggested using the normal map as a shallow, depth detail offset map, which I'll look a little more into. If any of you have some information on how to do this I'd be happy to learn more! Otherwise I'll just work my way from the discription Konrad gave me.
Anyways I'm glad you like the new blending, I was afraid that I would give you the wrong impression with my poor programmers-art heightmaps :P I'll keep you updated!
#7
06/26/2014 (9:44 am)
Even if it will be not so good, it is still valuable as an option, some textures may look better the old way, while others look better with the new method. 
#8
06/26/2014 (10:20 am)
Thats a nice improving at the terrain! 
#9
06/27/2014 (7:17 am)
I agree add this to the stock T3D great work Lukas!!!
#10
Before / After
It was done with very simple math, I basically just calculated the dot product between the horizontal plane and the normal in the texture and multiplied it to the blend value.
06/28/2014 (5:24 am)
So a little bit of progress, I did what Konrad recommended and used the normal map as a shallow depth detail map, the difference is noticeable but minimal:Before / After
It was done with very simple math, I basically just calculated the dot product between the horizontal plane and the normal in the texture and multiplied it to the blend value.
float4 bumpNormal = tex2D(normalMap0, IN.detCoord0.xy);
bumpNormal.xyz = bumpNormal.xyz * 2.0 - 1.0;
float normAngle = 0;
normAngle = dot( bumpNormal, float3(0,0,1) );
//...
ma = max(detailColor.a + detailBlend0, currentAlpha + invBlend) * normAngle - blendDepth;
#11
Funny enough I actually liked the mix of lerp'ed normals and blended textures more than using the same blending method, but I recon that it is only in this particular case and that it would be better for general use not to have the mix.
Before / After
So with these last changes, it's just about done... No more known issues! Nothing much more to do, except if we want to add some more features.
06/28/2014 (6:29 am)
More progress! Fixed the normal blending and fixed an issue where alpha values of 0 would result in the previous texture being dominant even if it was too far away to blend in.Funny enough I actually liked the mix of lerp'ed normals and blended textures more than using the same blending method, but I recon that it is only in this particular case and that it would be better for general use not to have the mix.
Before / After
So with these last changes, it's just about done... No more known issues! Nothing much more to do, except if we want to add some more features.
#12
You can see the issue in the below video where I also play around with the different parameters, and show how the blending generally looks.
Edit: sorry for bad video quality, my video recording settings must be messed up
06/28/2014 (6:42 am)
Ah would seem I was a bit too fast on the trigger there, there is an issue with some areas where the blending has "hard edges" I'll investigate this issue and try to solve it before cleaning up the code and prep for release.You can see the issue in the below video where I also play around with the different parameters, and show how the blending generally looks.
Edit: sorry for bad video quality, my video recording settings must be messed up
#13
Now I'm just down to adding some features.
There is this thing called "blendDepth" that allows you to tweak how sharp the blending it is, to show you the effect of tweaking this depth I created the following album:
http://imgur.com/a/oae0O
First image is stock T3D blending, from there it is new blending with blendDepth 0.1,0.2 etc up to 0.9. Last image is blendDepth 0.99
Please tell me what you think!
Edit: forgot to tell you what needs to be done before release!
Basically I'm going to try to make that blendDepth a per-texture parameter and see how it works out..
07/01/2014 (7:58 am)
Okay, so I believe all the bugs have been hunted down (even fixed a stock T3D blending bug and PR'ed it :P)Now I'm just down to adding some features.
There is this thing called "blendDepth" that allows you to tweak how sharp the blending it is, to show you the effect of tweaking this depth I created the following album:
http://imgur.com/a/oae0O
First image is stock T3D blending, from there it is new blending with blendDepth 0.1,0.2 etc up to 0.9. Last image is blendDepth 0.99
Please tell me what you think!
Edit: forgot to tell you what needs to be done before release!
Basically I'm going to try to make that blendDepth a per-texture parameter and see how it works out..
#14
07/01/2014 (9:04 am)
That looks awesome Lukas! So we'll be able to set the blendDepth?
#15
So you can have some textures that has a blendDepth of 0.9, and some that has 0.3 or whatever you want!
07/01/2014 (9:09 am)
@Scot, yeah it'll be a variable on the material, like this:new TerrainMaterial()
{
diffuseMap = "art/terrains/Example/sand";
diffuseSize = "500";
detailMap = "art/terrains/Example/sand_d";
detailSize = "2";
detailDistance = "50";
internalName = "sand2";
normalMap = "art/terrains/Example/sand_n";
blendDepth = 0.5;
};So you can have some textures that has a blendDepth of 0.9, and some that has 0.3 or whatever you want!
#16
EDIT: "even fixed a stock T3D blending bug and PR'ed it"
What was the existing blending bug you fixed?
07/01/2014 (9:26 am)
Will it be a setting in the material editor? The less people/artists have to go into script the better.EDIT: "even fixed a stock T3D blending bug and PR'ed it"
What was the existing blending bug you fixed?
#18
Make sure to download and test the PR if you have the time :)
07/01/2014 (3:34 pm)
Pull request! https://github.com/GarageGames/Torque3D/pull/712 Make sure to download and test the PR if you have the time :)
Duion
Because of the heightmap, if you want parallax you will need a heightmap in your normal map anyway, so this should not be the problem, you could use the heightmap that is already in there.
I don't know if an autocalculated version is necessary, if someone really wants this feature he can take the 5 mins to make a heightmap for his texture, this would also enable parallax, by the way, how does the blending work with parallax?
Otherwise you could just use the brightness of the texture, since dark spots are most times lower anyway.