T3D's Terrain Texturing "Concept"
by Mitovo · in Torque 3D Beginner · 08/15/2013 (9:07 pm) · 16 replies
Hello everyone,
Posting this as someone coming back to give T3D another go.
There are two things that I consider my nemeses with T3D, both related to the same feature. They are Terrain Sculpting and Terrain Texturing.
Terrain Sculpting is fodder for a different topic, as I want to focus on the texturing in this one.
In a nutshell, I'm having a lot of trouble understanding the "concept" behind how T3D's terrain texturing works. Actually, that's not a good way of putting it. I understand the concept behind it; I'm familiar with Diffuse Textures, Detail Textures, Normal Maps, and so on.
What trips me up, though, is that T3D seems to approach their use quite differently from any other engine or editor I've ever played with. I say this because if I use the approach to texturing in T3D that I use in any other engine (and that works fine in those cases), I get consistently ugly results and no amount of tweaking "strength" or "scale" values seems to make any difference. The results just go from one kind of ugly to another.
What I'm ultimately interested in is some kind of a guide, or bible, or comprehensive tutorial on:
- The "philosophy" or "concept" behind how they work
- The intended/required/recommended approach to selecting and using Diffuse maps.
- The intended/required/recommended approach to selecting and using Macro detail maps
- The intended/required/recommended approach to selecting and using "Micro" detail maps.
- Are there certain ranges of color or value or resolution recommended for any of the 3 image types, and why?
Basically "How did the GG programmers intend this system to work when they designed and programmed it". I'm assuming there was some certain process and approach they designed it around that would pretty much be consistent no matter what type of terrain was being created.
I'm sure there's a method to the madness, and I know that others have figured it out; I've seen some gorgeous work done with the tools. I just can not seem to crack that nut and figure it out myself.
I've tried searching, but no dice... Nothing that really explains the terrain texturing system on that kind of "nuts and bolts" level of the "what", "how" and most importantly (for me), the "why".
Any hope of something like this existing, whether now or in the future?
Thanks for your time :)
Posting this as someone coming back to give T3D another go.
There are two things that I consider my nemeses with T3D, both related to the same feature. They are Terrain Sculpting and Terrain Texturing.
Terrain Sculpting is fodder for a different topic, as I want to focus on the texturing in this one.
In a nutshell, I'm having a lot of trouble understanding the "concept" behind how T3D's terrain texturing works. Actually, that's not a good way of putting it. I understand the concept behind it; I'm familiar with Diffuse Textures, Detail Textures, Normal Maps, and so on.
What trips me up, though, is that T3D seems to approach their use quite differently from any other engine or editor I've ever played with. I say this because if I use the approach to texturing in T3D that I use in any other engine (and that works fine in those cases), I get consistently ugly results and no amount of tweaking "strength" or "scale" values seems to make any difference. The results just go from one kind of ugly to another.
What I'm ultimately interested in is some kind of a guide, or bible, or comprehensive tutorial on:
- The "philosophy" or "concept" behind how they work
- The intended/required/recommended approach to selecting and using Diffuse maps.
- The intended/required/recommended approach to selecting and using Macro detail maps
- The intended/required/recommended approach to selecting and using "Micro" detail maps.
- Are there certain ranges of color or value or resolution recommended for any of the 3 image types, and why?
Basically "How did the GG programmers intend this system to work when they designed and programmed it". I'm assuming there was some certain process and approach they designed it around that would pretty much be consistent no matter what type of terrain was being created.
I'm sure there's a method to the madness, and I know that others have figured it out; I've seen some gorgeous work done with the tools. I just can not seem to crack that nut and figure it out myself.
I've tried searching, but no dice... Nothing that really explains the terrain texturing system on that kind of "nuts and bolts" level of the "what", "how" and most importantly (for me), the "why".
Any hope of something like this existing, whether now or in the future?
Thanks for your time :)
About the author
#2
And I'd listen to Dan whenever he has anything to say about painting Torque terrains - his work is freakin' fantastic.
08/16/2013 (10:07 am)
I'm with Dan on that first point - whatever the initial idea was, that was 15 years ago and long before anyone had established much in the way of standard practices.And I'd listen to Dan whenever he has anything to say about painting Torque terrains - his work is freakin' fantastic.
#3
Thanks for the well-written response, and all that typing, Dan!
At least now you can simply link back to that post as a reference should anyone else ask the same, or similar question :). It's now a reusable resource!
That said, I think I maybe didn't explain what I meant when I wondered what GG's "concept" was. So, I'll elaborate. As I explained, I've used other terrain editors in other engines, stand-alone terrain editing programs (a lot of them). Almost all those programs seemed to base their tools around the same basic approach or process. As an analogy, if you've become familiar with playing Unreal Tournament, and then hop over to Quake 3... the controls are going to act and react pretty much the same. Moving over to another FPS would likely yield the same results. There's a consistency across those games, as they're all based around the same basic gameplay mechanics, etc. You press 'W' you're gonna run forward. Left-click, you're going to fire, etc... (unless you change default settings, of course :p)
To bring that analogy back to terrain editing, all the terrain editors I've used in the past (well, most all) have shared similar editing/painting mechanics. Being familiar with one editor made it pretty straight forward to jump into another because the basic process was the same, and the results were pretty consistent.
However, T3D sorta 'interrupts' that continuity as the familiar methods I've applied elsewhere don't get the same results. They're vastly different, in fact. So, it appears there's _something_ different going on "under the hood" in how GG handles their diffuse and detail layers, etc. And, so when I say "the concept", I mean "the way they've chosen to handle those things in their terrain engine", which in turn affects how the user works with them to get the desired result.
Long-winded explanation.. but there ya go lol.
It seems to me that T3D's setup is a bit more nuanced and provides more direct control by the editor/user.
Dan's done a good job of explaining the purpose of the 'Macro' detail layer. That bit I was a bit confused about, as I'm not used to having more than 1 detail layer to work with. It seems almost like the setup is something like a manual mip-mapping setup, where the transition points are defined by the user, not hard-coded into the engine/renderer.
So, that said, I am left with one more question... Is there a certain "optimal" value-range, or even image format I should be working in? I mean value of the image itself, as in "HSL", etc.. I find that when I apply a detail map, it either makes the surface too dark (like using a "color burn" blend at 100%, or overbright where it gets washed out. And no value in-between quite works either. I'm thinking there must be some specific value or something that I'm not using. I've tried looking that up, but nothing's really come back for that either.
Anyhoo..
Thanks again for the help/feedback!
It's much appreciated.
08/16/2013 (10:28 am)
Hey folks..Thanks for the well-written response, and all that typing, Dan!
At least now you can simply link back to that post as a reference should anyone else ask the same, or similar question :). It's now a reusable resource!
That said, I think I maybe didn't explain what I meant when I wondered what GG's "concept" was. So, I'll elaborate. As I explained, I've used other terrain editors in other engines, stand-alone terrain editing programs (a lot of them). Almost all those programs seemed to base their tools around the same basic approach or process. As an analogy, if you've become familiar with playing Unreal Tournament, and then hop over to Quake 3... the controls are going to act and react pretty much the same. Moving over to another FPS would likely yield the same results. There's a consistency across those games, as they're all based around the same basic gameplay mechanics, etc. You press 'W' you're gonna run forward. Left-click, you're going to fire, etc... (unless you change default settings, of course :p)
To bring that analogy back to terrain editing, all the terrain editors I've used in the past (well, most all) have shared similar editing/painting mechanics. Being familiar with one editor made it pretty straight forward to jump into another because the basic process was the same, and the results were pretty consistent.
However, T3D sorta 'interrupts' that continuity as the familiar methods I've applied elsewhere don't get the same results. They're vastly different, in fact. So, it appears there's _something_ different going on "under the hood" in how GG handles their diffuse and detail layers, etc. And, so when I say "the concept", I mean "the way they've chosen to handle those things in their terrain engine", which in turn affects how the user works with them to get the desired result.
Long-winded explanation.. but there ya go lol.
It seems to me that T3D's setup is a bit more nuanced and provides more direct control by the editor/user.
Dan's done a good job of explaining the purpose of the 'Macro' detail layer. That bit I was a bit confused about, as I'm not used to having more than 1 detail layer to work with. It seems almost like the setup is something like a manual mip-mapping setup, where the transition points are defined by the user, not hard-coded into the engine/renderer.
So, that said, I am left with one more question... Is there a certain "optimal" value-range, or even image format I should be working in? I mean value of the image itself, as in "HSL", etc.. I find that when I apply a detail map, it either makes the surface too dark (like using a "color burn" blend at 100%, or overbright where it gets washed out. And no value in-between quite works either. I'm thinking there must be some specific value or something that I'm not using. I've tried looking that up, but nothing's really come back for that either.
Anyhoo..
Thanks again for the help/feedback!
It's much appreciated.
#4
@Mike - You could always grab that terrain from the thread that Richard linked to get an idea of how things work (albeit on a very simplistic level). The overdark thing. Man, it's been so long since I had that same problem I can't even remember what caused it. It does come up from time to time though, so I can understand the headscratching it must be causing you.
While I can't remember exactly what it is that causes it, I have a feeling it's related to the compounded hue/saturation/contrast thing. More specifically, your textures (detail and macro) need to be above a certain RGB level or else they will blacken your diffuse map. I think.
Here are some conventions that you should know, however:
Always use a power-of-two ratio in your textures. You can use non Po2 but you'll get artifacts.
Formats you can use (that I know of at least) are .png, .dds and .jpg - if you're trying to use .tga or .tif then you're going to be very unhappy. I also believe that .bmp is not supported for use on the terrain, but again, I may be mistaken.
Edit: The thread you want perhaps is this one here. That should get you going.
08/16/2013 (11:08 am)
@Richard - Thanks mate, for both the compliment and for linking that demo terrain. I didn't even think to link that in here myself.@Mike - You could always grab that terrain from the thread that Richard linked to get an idea of how things work (albeit on a very simplistic level). The overdark thing. Man, it's been so long since I had that same problem I can't even remember what caused it. It does come up from time to time though, so I can understand the headscratching it must be causing you.
While I can't remember exactly what it is that causes it, I have a feeling it's related to the compounded hue/saturation/contrast thing. More specifically, your textures (detail and macro) need to be above a certain RGB level or else they will blacken your diffuse map. I think.
Here are some conventions that you should know, however:
Always use a power-of-two ratio in your textures. You can use non Po2 but you'll get artifacts.
Formats you can use (that I know of at least) are .png, .dds and .jpg - if you're trying to use .tga or .tif then you're going to be very unhappy. I also believe that .bmp is not supported for use on the terrain, but again, I may be mistaken.
Edit: The thread you want perhaps is this one here. That should get you going.
#5
A few more things about macro textures. They do need to tile, but they don't need to tile as well as the detail texture. By that I mean since macro textures are meant to be used with a large scale, features in the image that would make tiling obvious aren't a problem. I like keeping macro strength below the detail strength as well.
Here's an example macro texture I've made:
and an example image with macro textures in it. Note the large pattern in the sandy area below the Terrain Material Editor popup - that is in the macro, not the detail. Sorry for the giant image.
08/16/2013 (11:16 am)
As far as optimal images for detail and macro maps goes, look at your histograms. For both of them I like to use greyscale (or at least very desaturated) images with a nicely centered histogram. I try to avoid pure black or blown out areas. I do this for all of my detail textures, keeping them consistent so that I don't have to be playing with material strengths just because I changed a texture.A few more things about macro textures. They do need to tile, but they don't need to tile as well as the detail texture. By that I mean since macro textures are meant to be used with a large scale, features in the image that would make tiling obvious aren't a problem. I like keeping macro strength below the detail strength as well.
Here's an example macro texture I've made:
and an example image with macro textures in it. Note the large pattern in the sandy area below the Terrain Material Editor popup - that is in the macro, not the detail. Sorry for the giant image.
#6
08/16/2013 (11:25 am)
Woah, that's beautiful! 
#7
Also, I mix up the layers alot, scatter the details around the same areas to break up any repetition of the pattern, and feel it gives a more natural look to things myself if done with a bit of work
08/16/2013 (11:31 am)
Macro and Detail textures multiply with eachother too, if you have two of the same texture for detail and macro, around the player it can really strengthen the contrast of the detail texture as its being multiplied by the same texture underneath.Also, I mix up the layers alot, scatter the details around the same areas to break up any repetition of the pattern, and feel it gives a more natural look to things myself if done with a bit of work
#8
Oh, by the way - I should have added about that both the detail and macro textures act like an overlay layer in photoshop/gimp. You could use your diffuse as a background image and set your detail and macro on new layers over it set to overlay and see what is going on. That's why having a nice centered histogram is important - middle grey doesn't do much with overlay, while your highlights and shadows do.
Also, feel free to use that texture. I consider it public domain.
08/16/2013 (11:48 am)
Dan and Andy Wright are the guys who have shared so much to make stuff like that possible. Torque can make great big pretty terrains that run really well, and the current tool layout and official documentation have obfuscated that.Oh, by the way - I should have added about that both the detail and macro textures act like an overlay layer in photoshop/gimp. You could use your diffuse as a background image and set your detail and macro on new layers over it set to overlay and see what is going on. That's why having a nice centered histogram is important - middle grey doesn't do much with overlay, while your highlights and shadows do.
Also, feel free to use that texture. I consider it public domain.
#9
Learnt a lesson last time, dont ever think someone will stick to a 3 year old agreement by email... I sent him back what he said to me and had no reply, bravo... anyway lol, lesson learned but I'll have a site up in the coming weeks dedicated to free terrains, materials, environmental effects and tutorials, I have a new source for base photography that is happy for me to use them for this project as it will be adding seamless content to his site too in return.
I Have a bit more work to do sorting out content, normals/displacements/materials.cs scripts etc, im back upto a 100 material library, have to retexture my terrains but a barebones site is ready and waiting for it when I have it all together, it all feels very much like a hobby I'm enjoying again instead of feeling a reluctance inside me to work on stuff because it started feeling like a job... the enthusiasm is back.
edit:(must stop derailing threads, sorry)
08/16/2013 (12:24 pm)
Ive always gone with a mix of highpass and manual desaturation. unfortunately you cant just batch every material with the same settings etc because one come out looking great... its very dependant on the base image...Learnt a lesson last time, dont ever think someone will stick to a 3 year old agreement by email... I sent him back what he said to me and had no reply, bravo... anyway lol, lesson learned but I'll have a site up in the coming weeks dedicated to free terrains, materials, environmental effects and tutorials, I have a new source for base photography that is happy for me to use them for this project as it will be adding seamless content to his site too in return.
I Have a bit more work to do sorting out content, normals/displacements/materials.cs scripts etc, im back upto a 100 material library, have to retexture my terrains but a barebones site is ready and waiting for it when I have it all together, it all feels very much like a hobby I'm enjoying again instead of feeling a reluctance inside me to work on stuff because it started feeling like a job... the enthusiasm is back.
edit:(must stop derailing threads, sorry)
#10
Is that made from sattelite photos? Thanks for all the little tips and hints, that's just awesome.
08/16/2013 (3:17 pm)
I really like how that terrain looks quite realistic, at least if you look further away than a few meters.Is that made from sattelite photos? Thanks for all the little tips and hints, that's just awesome.
#11
08/16/2013 (4:12 pm)
@Stefan: if you are talking about the large image I posted above, the diffuse texture and terrain were completely generated by World Machine. The diffuse is set to be the same size as the terrain to match it perfectly. Every material uses the same diffuse with the same settings.
#12
I guess that makes the diffusemap quite large. And the detail maps gives it the detailed look up close? Sounds reasonable. Thanks!
08/16/2013 (4:22 pm)
I haven't heard of World Machine before. How does it compare to Terragen?I guess that makes the diffusemap quite large. And the detail maps gives it the detailed look up close? Sounds reasonable. Thanks!
#13
Each terrain material layer uses the same diffuse, but has different detail/macro layers on it. the diffuse is for long distance detail and a uniformity to the colouring throughout the areas, and detail maps are used for the fidelity around the player, with macro for mid range.
a very scruffy quickly made tutorial on the second page of this thread
http://www.garagegames.com/community/forums/viewthread/120247
08/16/2013 (5:24 pm)
the diffuses for that method are usually 2048 or 4096, I use the same method myself with maps from geocontrol2Each terrain material layer uses the same diffuse, but has different detail/macro layers on it. the diffuse is for long distance detail and a uniformity to the colouring throughout the areas, and detail maps are used for the fidelity around the player, with macro for mid range.
a very scruffy quickly made tutorial on the second page of this thread
http://www.garagegames.com/community/forums/viewthread/120247
#14
But man, things have changed. With this approach, I guess manual modifications are out of question? That would make the diffuse out of sync with the heightmap, right?
Guess I'll have to find out. :) Thanks again!
08/16/2013 (7:28 pm)
Excellent tutorial!But man, things have changed. With this approach, I guess manual modifications are out of question? That would make the diffuse out of sync with the heightmap, right?
Guess I'll have to find out. :) Thanks again!
#15
08/17/2013 (6:24 pm)
Not entirely, ive sometimes created multiple diffuses like that for the same heightmap, then merged them together to create differences in areas... was more experimentation on an idea but worked fine on a large enough terrain 
#16
First use same texture for base,macro and detail
if it does not look right (in cases of very colorful textures) desaturate the texture and use the desaturated-one for macro and detail, does not have to be 100% desaturated, you can leave in a little color for additional variation or detail
if you get too dark or too bright areas adjust the brightness and contrast until it fits
if it still looks ugly (in most cases where patterns are visible in a too big scale, for example big grass blades on a base or macro map, that appear when viewed from far away) blur them out in the base texture or destroy the pattern otherwise so it is no longer identifiable or just play with the scaling in base and macro till it looks right
if macro map still does not look good, some tips are to use your blurred or otherwise modified base map, only in a desaturated way or just rendered clouds or noise on a picture will also work fine and will add some variation
if everything is okay generate normal with heightmap from detail map
if not, repeat other steps until it is okay
08/22/2013 (2:34 am)
I use some kind of algorithm to make terrain materials.First use same texture for base,macro and detail
if it does not look right (in cases of very colorful textures) desaturate the texture and use the desaturated-one for macro and detail, does not have to be 100% desaturated, you can leave in a little color for additional variation or detail
if you get too dark or too bright areas adjust the brightness and contrast until it fits
if it still looks ugly (in most cases where patterns are visible in a too big scale, for example big grass blades on a base or macro map, that appear when viewed from far away) blur them out in the base texture or destroy the pattern otherwise so it is no longer identifiable or just play with the scaling in base and macro till it looks right
if macro map still does not look good, some tips are to use your blurred or otherwise modified base map, only in a desaturated way or just rendered clouds or noise on a picture will also work fine and will add some variation
if everything is okay generate normal with heightmap from detail map
if not, repeat other steps until it is okay
Torque Owner Dan Webb
PsyCandy Games
First of all, forget questioning whatever GarageGames' initial concept was for terrains. It doesn't mean anything to me, and I can't see it meaning much to you either, Mike. Let's start with an empty cup, shall we?
Let's begin with the diffuse map. This, in a nutshell is the image of the terrain seen from a distance. In fact, let's go ahead and call it the distance material layer.
Distance Material Layer (diffuse)
This is the base upon which everything else is added onto. I say it like that because the hue/saturation/contrast values for every successive layer compounds on the ones below. For this reason, you really only need colours in this layer, as the screen won't render the required pixels for you to make out too much detail on it. This layer is best used at the same scale as the heightmap you're using, or the size of the terrain if generated within the terrain editor.
As you zoom in closer to the terrain, the first thing you'll notice is the macro texture, or medium material layer.
Medium Material Layer (macro)
This layer will give you another level of texture detail, starting from the distance applied in that material's settings all the way to zero, which is at the player's feet while standing on the ground. Because this is a mid-range material, it's not important to incorporate much detail in this texture, although you'll want more than is in the distance material layer (diffuse map). You can experiment a lot with hue/saturation/contrast settings in this layer, as it's quite forgiving to a point. It will really depend a lot on what detail texture, or near material layer that you're planning to use.
Near Material Layer (detail)
This is obviously the texture with the most detail, but which will also tile quite a lot. This tiling is the reason we have a medium material layer in the first place. Because it tiles, you want to have a reasonably short range falloff (50m by default), but that looks kinda weird when it blends straight into the lower detail of the diffuse map (distance material layer), so the macro, or medium material layer acts as a buffer of sorts.
Settings
I always use a single diffuse map which I have as the distance material layer of every consecutive material for that terrain. And it's always set to be the same size as the terrain. Note that enabling side projection on this layer does not affect this layer, but the detail layer, but because the diffuse/distance layer is a kind of parent layer the check box is here.
I then set my detail/near material layer before I bother with the macro, as I need to get a sense of falloff for this layer first. In general, I use a distance of 50 meters/units and a strength of between 0.5 and 0.75, depending on how dark this texture is, and how dark the macro/diffuse textures are. Size is usually between 3 and 5.
I never use the default settings for the macro texture, and sometimes I don't even use this at all. In general though, I set the size to 80, the strength to no more than 0.5 and tweak it if I need to from there.
I rarely use normal maps anymore. Basically I use normal maps when I really need to have the added relief on that part of the terrain. Namely, rock and rocky dirt.
I hope this helps you, as I never want to type this again. :D