Terrain Macro Texture
by Adam Armstrong · 02/27/2013 (11:40 am) · 31 comments
I had gone back and forth trying to find a good compromise between up close detail and distant terrain having any detail at all on it. Even with very large detail textures (2048 and even 4096) that tiled perfectly and unique 4096x4096 diffuse maps, I wasn't very happy with the middle distances in between the detail coverage and the far distance where the diffuse was sufficient. Just raising the detail distance was an affair of diminishing returns that eventually just added tiling artifacts.
For awhile I tried using very large detail map scales, on the order of 75-100m and a visible distance of around 1000m. I ran the detail textures through a set of artistic filters to help ignore the blurry mess of the up close textures. I decided that a mix of the two was called for, and decided to see how hard that would be.
I just made something I'm calling Macro detail that functions just like the detail texture, but without the normal/parallax being tied in with it. You select a texture, and set the distance and scale. I've been using values like 100m scale and 1000m distance. Performance is pretty good. My theory is that it is another blend on each pixel it touches, and if the card doesn't need to do another pass then there is no problem. When there are many terrain materials using it though, that extra blend adds up. In the example photos I have six materials all using detail + macro, and I'm getting about a 5-10 fps hit. Used more sparingly it isn't noticeable.
I'm afraid I ended up touching most of the files in engine/terrain. I didn't do a very good job of keeping track of what I was doing, and I was working with the stock 1.2 code. I'm going to have to buckle down and get this working in 2.0 and figure out how git works next. That might take a little time, but I'm eager to get this out there.
Anyways, example images. I'm using World Machine to make my terrain and diffuse texture, using a single diffuse for the entire terrain. I placed my terrain materials just using slopes values and Konrad Kiss's Rule Based Layer Distribution resource. Macro textures of course have to tile, but they don't have to be as uniform as a good detail texture - they are faking the effect of unique detail at a distance, and since they are so large tiling won't really be apparent.
To see all of these images, right click and select "view image" or however your browser deals with that.
For awhile I tried using very large detail map scales, on the order of 75-100m and a visible distance of around 1000m. I ran the detail textures through a set of artistic filters to help ignore the blurry mess of the up close textures. I decided that a mix of the two was called for, and decided to see how hard that would be.
I just made something I'm calling Macro detail that functions just like the detail texture, but without the normal/parallax being tied in with it. You select a texture, and set the distance and scale. I've been using values like 100m scale and 1000m distance. Performance is pretty good. My theory is that it is another blend on each pixel it touches, and if the card doesn't need to do another pass then there is no problem. When there are many terrain materials using it though, that extra blend adds up. In the example photos I have six materials all using detail + macro, and I'm getting about a 5-10 fps hit. Used more sparingly it isn't noticeable.
I'm afraid I ended up touching most of the files in engine/terrain. I didn't do a very good job of keeping track of what I was doing, and I was working with the stock 1.2 code. I'm going to have to buckle down and get this working in 2.0 and figure out how git works next. That might take a little time, but I'm eager to get this out there.
Anyways, example images. I'm using World Machine to make my terrain and diffuse texture, using a single diffuse for the entire terrain. I placed my terrain materials just using slopes values and Konrad Kiss's Rule Based Layer Distribution resource. Macro textures of course have to tile, but they don't have to be as uniform as a good detail texture - they are faking the effect of unique detail at a distance, and since they are so large tiling won't really be apparent.
To see all of these images, right click and select "view image" or however your browser deals with that.
About the author
#2
02/27/2013 (1:48 pm)
Good work! 
#3
02/27/2013 (5:28 pm)
Awesome! 
#4
02/27/2013 (5:45 pm)
Wow, looks really good! 
#5
Interesting work. I am wondering about Texture memory usage. For instance, if I run this system I know really huge terrain textures eat up a good portion of texture ram and are present in memory simply because the terrain would be always present. Again, I don't know all the ins and outs of how you accomplished these images *which are REALLY GOOD by the way. I am just concerned that developers get really good terrains and nothing else because of further limiting memory or other resources. This is a primary issue with Id tech 5. Texture memory gets overworked and leaves little for the rest of the game or world that is created. We won't get into the compression and texture pop issues with that engine. Just something to think about....
Ron
02/27/2013 (5:55 pm)
Adam,Interesting work. I am wondering about Texture memory usage. For instance, if I run this system I know really huge terrain textures eat up a good portion of texture ram and are present in memory simply because the terrain would be always present. Again, I don't know all the ins and outs of how you accomplished these images *which are REALLY GOOD by the way. I am just concerned that developers get really good terrains and nothing else because of further limiting memory or other resources. This is a primary issue with Id tech 5. Texture memory gets overworked and leaves little for the rest of the game or world that is created. We won't get into the compression and texture pop issues with that engine. Just something to think about....
Ron
#6
Point taken about texture memory. I think what I'm doing here is very manageable in that regard. My modification just adds another detail texture. It can be any valid size. If you were using a diffuse, detail, and normal texture on a terrain material before, this would add one more texture to that load. Even for the 2048x2048 detail textures I'm using, that's less than 2 MB per texture. A few of my terrain materials use the same texture as a macro and detail texture, which ideally doesn't add anything at all. I don't know enough about texture handling to guarantee that right now, so I won't make that claim just yet.
If people are interested in playing with a rough version of this I could scrape something together - a little bundle of source, an .exe, and the gui's I've changed. I suppose I should learn about github at some point.
02/27/2013 (8:12 pm)
Ron,Point taken about texture memory. I think what I'm doing here is very manageable in that regard. My modification just adds another detail texture. It can be any valid size. If you were using a diffuse, detail, and normal texture on a terrain material before, this would add one more texture to that load. Even for the 2048x2048 detail textures I'm using, that's less than 2 MB per texture. A few of my terrain materials use the same texture as a macro and detail texture, which ideally doesn't add anything at all. I don't know enough about texture handling to guarantee that right now, so I won't make that claim just yet.
If people are interested in playing with a rough version of this I could scrape something together - a little bundle of source, an .exe, and the gui's I've changed. I suppose I should learn about github at some point.
#7
If you go over your pipeline in World Machine (very basically) it'll help to nail down any issues or complications you have. I'm thinking that you can get huge visual improvements here just by mastering WM's texturing tools.
Feel free to hit me with ideas if you want. I'm always up for a chat with a fellow terrain guy. :)
02/27/2013 (8:28 pm)
Adam, are you using WM's Basic Coverage macro as the base for your diffuse texture? Looks to me like your detail loss is due more to your diffuse map than anything. If you go over your pipeline in World Machine (very basically) it'll help to nail down any issues or complications you have. I'm thinking that you can get huge visual improvements here just by mastering WM's texturing tools.
Feel free to hit me with ideas if you want. I'm always up for a chat with a fellow terrain guy. :)
#8
If I hadn't been foolish enough to overwrite the .TMD for this map, I'd share that with you. As I recall, it isn't using the basic coverage macro, although the techniques are somewhat similar. I also fooled around in Photoshop using height and slope masks from World Machine to apply some texture (I don't have the 64 bit version of world machine, and since a map this size is at the limits of what I can do right now Photoshop is faster for things like that).
I'm very ready to believe my programmer art has some problems, but I'm not sure how any diffuse can make up the difference I'm seeing. My examples have a 4096x4096 diffuse on a 2048x2048 terrain with a 4 meter square size - a 8192x8192m terrain. The diffuse is 2 meters per pixel. My macro textures in the scene vary from 75-200 meters for a 2048x2048 texture, conservatively giving me something around a tenth of a meter or better. An added bonus is that since my detail textures don't have to work as hard, the distance on them goes way down (~50m in this scene) and I'm getting around a 5mm pixel size on them. Even without pushing map and texture sizes like I am, improvements are pretty easy to see with this modification.
Sorry for the enormous images, but we are talking about visual quality here. I've dropped the macro from this first shot and extended the detail texture distances by a factor of 10. While I'm still happy with the distant hills and haven't compromised on up close detail, the middle ground is blurry. The detail isn't hiding those big fat 2 meter pixels.
Here's the same scene with the macro texture working. An added benefit is that since I have more textures repeating at different frequencies (by making sure to use varied texture sizes) I get fake details that arise from textures overlapping in interesting ways. The foreground looks more varied, and tiling becomes even harder to detect.
I do intend to keep improving my art, and I'm very interested in seeing how others are using World Machine - the forums over there are not very helpful in that regard. I've only seen a few examples from people working on real time engines, and those are pretty much for UDK or CryEngine 2/3.
02/27/2013 (9:36 pm)
Dan,If I hadn't been foolish enough to overwrite the .TMD for this map, I'd share that with you. As I recall, it isn't using the basic coverage macro, although the techniques are somewhat similar. I also fooled around in Photoshop using height and slope masks from World Machine to apply some texture (I don't have the 64 bit version of world machine, and since a map this size is at the limits of what I can do right now Photoshop is faster for things like that).
I'm very ready to believe my programmer art has some problems, but I'm not sure how any diffuse can make up the difference I'm seeing. My examples have a 4096x4096 diffuse on a 2048x2048 terrain with a 4 meter square size - a 8192x8192m terrain. The diffuse is 2 meters per pixel. My macro textures in the scene vary from 75-200 meters for a 2048x2048 texture, conservatively giving me something around a tenth of a meter or better. An added bonus is that since my detail textures don't have to work as hard, the distance on them goes way down (~50m in this scene) and I'm getting around a 5mm pixel size on them. Even without pushing map and texture sizes like I am, improvements are pretty easy to see with this modification.
Sorry for the enormous images, but we are talking about visual quality here. I've dropped the macro from this first shot and extended the detail texture distances by a factor of 10. While I'm still happy with the distant hills and haven't compromised on up close detail, the middle ground is blurry. The detail isn't hiding those big fat 2 meter pixels.
Here's the same scene with the macro texture working. An added benefit is that since I have more textures repeating at different frequencies (by making sure to use varied texture sizes) I get fake details that arise from textures overlapping in interesting ways. The foreground looks more varied, and tiling becomes even harder to detect.
I do intend to keep improving my art, and I'm very interested in seeing how others are using World Machine - the forums over there are not very helpful in that regard. I've only seen a few examples from people working on real time engines, and those are pretty much for UDK or CryEngine 2/3.
#9
02/28/2013 (6:34 am)
Awesome, looking pretty good! 
#11
02/28/2013 (5:06 pm)
Adam: Fantastic! I'd be curious to see what an external fps app like FRAPS would show with and without your mod. It's nice to see terrain that looks like Cry3 or UDK. Now if only we could get around the collision/tiling issues... 
#13
03/01/2013 (11:02 pm)
Adam, I see the result a lot more clearly now. That's a huge difference! So, how exactly did you go about doing this. if I might ask? 
#14
That's one of the best looking terrains I've seen done. Great job!
As Dan mention do you care to release how this is done.
03/03/2013 (2:26 pm)
AdamThat's one of the best looking terrains I've seen done. Great job!
As Dan mention do you care to release how this is done.
#15
I am going to release this as soon as I bring it up to date. I started on it pretty casually with my copy of 1.2, and was surprised when I got it working in a few days. I've gotten myself a github account and have set up a fork to bring the changes I've made over. Once that is working, I'll let people play with it before putting in a pull request. Please be patient - the day job, side job, and two-month old infant have priority. I expect it to be a week or two just yet.
I decided I wanted something that worked just like the detail layer, but would take different sizes and distances. I just went into the engine and basically duplicated a fair amount of stuff where the detail texture is dealt with, leaving out the bits for a normal/parallax map. It is possible that someone would want that functionality I suppose, but I was guessing the performance would be prohibitive. I'm hoping I've done an efficient job of it - I have programmer art and artist programming.
03/03/2013 (10:09 pm)
Thanks everyone for the positive comments.I am going to release this as soon as I bring it up to date. I started on it pretty casually with my copy of 1.2, and was surprised when I got it working in a few days. I've gotten myself a github account and have set up a fork to bring the changes I've made over. Once that is working, I'll let people play with it before putting in a pull request. Please be patient - the day job, side job, and two-month old infant have priority. I expect it to be a week or two just yet.
I decided I wanted something that worked just like the detail layer, but would take different sizes and distances. I just went into the engine and basically duplicated a fair amount of stuff where the detail texture is dealt with, leaving out the bits for a normal/parallax map. It is possible that someone would want that functionality I suppose, but I was guessing the performance would be prohibitive. I'm hoping I've done an efficient job of it - I have programmer art and artist programming.
#16
But getting rid of the blurry textures is one thing on top of my wishlist. The weak point on torque is terrain textures on medium range.
03/09/2013 (6:06 pm)
Since I know how to use parallax map I parallax everything, everywhere, I did not find out how much this affects performance so far.But getting rid of the blurry textures is one thing on top of my wishlist. The weak point on torque is terrain textures on medium range.
#17
The diffuse is the main source of detail on the mid and long range textures and going upto 4096 greatly improves the detail. it also quadruples the ram usage though for that layer...
03/23/2013 (7:59 am)
the default diffuse texture still gets knocked down to a 2048 no matter what original size you use, when torque processes the dds for the base terrain diffuse it will go down to 2048, theres a 1 line change resource that i put up that allows a 4096 base diffuse.The diffuse is the main source of detail on the mid and long range textures and going upto 4096 greatly improves the detail. it also quadruples the ram usage though for that layer...
#18
03/24/2013 (10:29 am)
I'm curious, what's the memory consumption look like for this?
#19
06/16/2013 (4:33 am)
I wonder how you created the macro texture, do you have any concepts about how to create a macro texture? In some cases you just can take the detail texture again or any other detail texture, but for many things this does not look authentic.
#20
06/16/2013 (6:07 am)
Actually, it seems to be all about hue and saturation. If I place a medium green macro texture image together with the same hue and saturation detail texture, it effectively doubles the saturation value for that hue. In this case, a complimentary colour can work (somewhat) but I find that the detail or the macro texture needs to be desaturated. Doesn't seem to matter which. I also noticed a huge visual improvement by making the detail strength slightly higher than the macro strength. Example: macro = 0.5 and detail = 0.75 creates an almost perfect material. With my textures, at any rate.
Torque Owner Daniel Buckmaster
T3D Steering Committee