How Torque 3D is graphically challenged
by Bryce · in Torque 3D Professional · 04/06/2013 (5:01 pm) · 21 replies
Hey guys, I thought I'd bring up an issue that has been really bothering me about the Torque 3D engine. It's the tone map support. We've got amazing tools at our disposal when it comes to it: PureLight, for one, and whatever our 3D modeler has for baking.
Despite this, suddenly our materials look flat and limited when you give them a tone map! No normal map support. Specular settings are suddenly entirely hit-or-miss and show up in hideous circles. Without a tone map, everything shows up just fine. You can give a material specular highlighting, a normal map, and enable parallex to create the most stunning environments, and then say goodbye to it all the second you decide you want the lighting to look better.
We've got the right lightmapping tools. So what's stopping us from creating a scene like this?

Can anyone help me work with the material system so that tone mapping doesn't limit Torque's artistic potential? There's no reason a tone mapped material shouldn't be allowed to have proper speculars and a normal map.
Cheers,
-Bryce
Despite this, suddenly our materials look flat and limited when you give them a tone map! No normal map support. Specular settings are suddenly entirely hit-or-miss and show up in hideous circles. Without a tone map, everything shows up just fine. You can give a material specular highlighting, a normal map, and enable parallex to create the most stunning environments, and then say goodbye to it all the second you decide you want the lighting to look better.
We've got the right lightmapping tools. So what's stopping us from creating a scene like this?

Can anyone help me work with the material system so that tone mapping doesn't limit Torque's artistic potential? There's no reason a tone mapped material shouldn't be allowed to have proper speculars and a normal map.
Cheers,
-Bryce
#2
Adding a tone texture before is not very useful, because if you rotate the model, the shadow will be on the wrong side for example.
04/07/2013 (11:25 am)
In other engines the lightmaps are calculated inside the engine and then saved to a file, I believe.Adding a tone texture before is not very useful, because if you rotate the model, the shadow will be on the wrong side for example.
#3
The materials can be referenced the same way that datablock names are when creating server ghosted client side objects are doing now when it comes to using the materials on ghosted objects when it comes to making the implementation change.
04/07/2013 (1:19 pm)
If it indeed is a networking issue with syncing and sending materials from server to client as Richard Ranft believes then sounds like it's time to make materials not networkable in regards to ghosting them to client side, but instead require the materials to already exist on client side for implementation purposes to workaround this limitation.The materials can be referenced the same way that datablock names are when creating server ghosted client side objects are doing now when it comes to using the materials on ghosted objects when it comes to making the implementation change.
#4
Anyway, making materials client-side only sets up its own limitations. Perhaps a switch or something so that you can select the desired behavior. That way people who are relying on the networked behavior can continue to use it (the RPG guys) while people who don't want that can take advantage of the other behavior.
Actually, it would be better to adapt it to work correctly in either mode. I'm not certain what the limitation was but I'd guess it had to do with bandwidth optimization.
04/07/2013 (2:18 pm)
I'm just guessing....Anyway, making materials client-side only sets up its own limitations. Perhaps a switch or something so that you can select the desired behavior. That way people who are relying on the networked behavior can continue to use it (the RPG guys) while people who don't want that can take advantage of the other behavior.
Actually, it would be better to adapt it to work correctly in either mode. I'm not certain what the limitation was but I'd guess it had to do with bandwidth optimization.
#5
The way I opted to do it(realizing the issue Richard points out, where not all games need it), I just set up a $pref that can be set for the game so it decides if it loads the materials on the server or not.
Is this the sort of thing we're talking about here, or is it something different and I'm not reading this right?
04/08/2013 (6:00 am)
I dunno if we're talking about the exact same thing in regards to 'networked materials', but in order to get my material-based effects working(bullet effects defined by a 'physical material' datablock that's attached to a material of choice, etc), in order for it to correctly spawn the desired effects, some of which directly impacted gameplay such as bullet penetration, I got it where it'll generate everything on both client and server, and it works when networked.The way I opted to do it(realizing the issue Richard points out, where not all games need it), I just set up a $pref that can be set for the game so it decides if it loads the materials on the server or not.
Is this the sort of thing we're talking about here, or is it something different and I'm not reading this right?
#6
04/08/2013 (10:43 am)
Note Jondo's comments in this thread. I believed Tom's comments in this thread might be relevant, but upon reading it again I think not so much (though still informative).
#7
For the tonemap stuff, I'll admit, I haven't actually looked at it, as I hadn't any real inclination to use tone/light maps, but it raises a very valid point if you're looking at quality AND performance(especially with us on the precipice of having ios/android support from that one awesome user I forget his name at the moment.
Stuff like lightmaps would be worth making sure they work if possible.
I haven't looked at how the tonemaps are done, so it may be the same strain as the terrain specular(just not enough wiggleroom for the shaders to work) but one would need to poke through the shaders in question to see where the limitation lies. I'm fairly sure you could, in a pinch, trade one layer for another to get it all working at the same time.
04/08/2013 (11:19 am)
Yeah, I'm pretty sure we're looking at different issues there.For the tonemap stuff, I'll admit, I haven't actually looked at it, as I hadn't any real inclination to use tone/light maps, but it raises a very valid point if you're looking at quality AND performance(especially with us on the precipice of having ios/android support from that one awesome user I forget his name at the moment.
Stuff like lightmaps would be worth making sure they work if possible.
I haven't looked at how the tonemaps are done, so it may be the same strain as the terrain specular(just not enough wiggleroom for the shaders to work) but one would need to poke through the shaders in question to see where the limitation lies. I'm fairly sure you could, in a pinch, trade one layer for another to get it all working at the same time.
#8

04/08/2013 (2:59 pm)
Do you mean on a performance level for using tonemaps ? That lightings possible with dynamic point lights but granted they take up a bit more resources than a tonemap would, but tonemaps are static. {not a close scene granted, but see the pipework for the similar shadowing if thats what youre on about}
#9
Actually, that's an issue I've had with many games since around the time that Oblivion came out. Too much shiny. Shiny walls, shiny rocks, shiny dirt. The spec maps need to be toned way down IMO.
Perhaps a way to select on a per-material basis if you're going to use one or more of spec, parallax, tone, bump, etc. would help to better customise materials for both effect and performance.
04/08/2013 (3:06 pm)
Those shots are sweet - but, having worked in environments like that I can tell you that concrete is generally not that shiny.... ;pActually, that's an issue I've had with many games since around the time that Oblivion came out. Too much shiny. Shiny walls, shiny rocks, shiny dirt. The spec maps need to be toned way down IMO.
Perhaps a way to select on a per-material basis if you're going to use one or more of spec, parallax, tone, bump, etc. would help to better customise materials for both effect and performance.
#10
Alot of games like you say, since oblivion, seem to shrinkwrap everything in shiny plastic
04/08/2013 (3:12 pm)
yeah that got toned down once i did some spec maps for the textures and sorted out the normals so that painted brickwork wasnt lifted out so much.Alot of games like you say, since oblivion, seem to shrinkwrap everything in shiny plastic
#11
As you point out, much of the stuff that we do really depends on the situation. Point lights are more expensive than a tone map, but under certain circumstances they're preferable - so the rest of the area needs to be adjusted accordingly to avoid overloading the renderer.
04/08/2013 (3:16 pm)
You always turn out good stuff, so I figured that was WIP.As you point out, much of the stuff that we do really depends on the situation. Point lights are more expensive than a tone map, but under certain circumstances they're preferable - so the rest of the area needs to be adjusted accordingly to avoid overloading the renderer.
#12
04/08/2013 (3:21 pm)
lol thanks, havent done anything in quite a while, had health issues that took over life a bit too much. their pretty old those shots, was jus ttesting out arteria stuff in torque, hell i think it was possibly even one of the beta releases when they were done... still gotta get mit and get catched up on stuff :|
#13
You can even add a normal map and turn on speculars that use those point lights, but the specular effect is atrocious when you've got a tone map.
04/08/2013 (4:12 pm)
I usually bake a tone map, and then add in my point lights with "Represented in Lightmap" enabled. That way it'll light up players and other dynamic objects while leaving the tone mapped objects alone.You can even add a normal map and turn on speculars that use those point lights, but the specular effect is atrocious when you've got a tone map.
#14
I remember some old stuff (variables) which I don't think ever got done like canRecieveSunlight and canRecieveLM (or something like that - long since removed I believe).
I guess that in an ideal world that the mesh using a tonemap should recieve light for specular only - remember normals need spec to work or they just look flat - with diffuse not recieving light, and thus only using the tonemap.
To be honest, in my huge old level, I slapped low powered lights around - including some level wide ones in a trianglular crossfire - so that surfaces got some slight spec. And of course my lights were grouped so I could turn them on and off depending on where the player was - saved a ton of performance.
04/09/2013 (8:18 am)
Yeah "represented in lightmap" has always been a bit ... ewwwww. I avoided using it.I remember some old stuff (variables) which I don't think ever got done like canRecieveSunlight and canRecieveLM (or something like that - long since removed I believe).
I guess that in an ideal world that the mesh using a tonemap should recieve light for specular only - remember normals need spec to work or they just look flat - with diffuse not recieving light, and thus only using the tonemap.
To be honest, in my huge old level, I slapped low powered lights around - including some level wide ones in a trianglular crossfire - so that surfaces got some slight spec. And of course my lights were grouped so I could turn them on and off depending on where the player was - saved a ton of performance.
#15
04/09/2013 (1:42 pm)
It seems that when setting the Minnaert Constant to 1 for the materials with a tone map the strange specular circles vanish.
#16
When I first started using the "representedInLightmap" option on my lights, I had no idea how many problems it was causing. I interpreted it as "hey, don't cast light on lightmapped things," which is correct, but it comes with tons of horrible side effects.
Minaert constant always seemed like one of those settings you weren't supposed to touch. Richard is right, setting it to 1 on tonemapped materials stops the speculars from going wonky.
I decided to have a bit of fun recreating the Mirror's Edge screenshot I originally posted. Made it and baked the lightmap in Blender, and then used Steve and Richard's findings while I set up the materials: lights shouldn't use "representedInLightmap", and "minaert constant" should be changed from -1 to 1.
Suddenly, we get working speculars and everything looks great.






So it appears there is a way around all this nonsense. I really wish we didn't have to fumble around with all these settings to get something to look half decent with tonemaps.
To anyone reading this in the future who wants normals, specular, and tonemaps to work together:
1) Set material minaert constants to 1.
2) Don't use "representedInLightmap" in your point lights, just lower the brightness to around 0.3 - 0.5 or so.
04/09/2013 (6:24 pm)
Quote:Yeah "represented in lightmap" has always been a bit ... ewwwww. I avoided using it.
When I first started using the "representedInLightmap" option on my lights, I had no idea how many problems it was causing. I interpreted it as "hey, don't cast light on lightmapped things," which is correct, but it comes with tons of horrible side effects.
Quote:It seems that when setting the Minnaert Constant to 1 for the materials with a tone map the strange specular circles vanish.
Minaert constant always seemed like one of those settings you weren't supposed to touch. Richard is right, setting it to 1 on tonemapped materials stops the speculars from going wonky.
I decided to have a bit of fun recreating the Mirror's Edge screenshot I originally posted. Made it and baked the lightmap in Blender, and then used Steve and Richard's findings while I set up the materials: lights shouldn't use "representedInLightmap", and "minaert constant" should be changed from -1 to 1.
Suddenly, we get working speculars and everything looks great.






So it appears there is a way around all this nonsense. I really wish we didn't have to fumble around with all these settings to get something to look half decent with tonemaps.
To anyone reading this in the future who wants normals, specular, and tonemaps to work together:
1) Set material minaert constants to 1.
2) Don't use "representedInLightmap" in your point lights, just lower the brightness to around 0.3 - 0.5 or so.
#17
04/10/2013 (5:39 am)
Very nice work there!
#18
All other engines I know had the function to generate lightmaps inside the engine, so you build your level, place your lights, then press "generate lightmaps" , wait some time and all is done.
I'm wondering why Torque does not have that, is this so complicated?
04/10/2013 (6:17 am)
Is this with or without tone maps? I comparison would be nice. I think tonemaps are too complicated and a waste of space and resources. You cannot reuse objects/textures with baked tone maps, because under different circumstances it would look all wrong. So you need new models with new textures every scene and place.All other engines I know had the function to generate lightmaps inside the engine, so you build your level, place your lights, then press "generate lightmaps" , wait some time and all is done.
I'm wondering why Torque does not have that, is this so complicated?
#19
From there, you could probably do a dynamic hijack of materials for certain objects on-load and re purpose the tonemap field to use the level-generated ones.
Just add a field to statics like 'useLightmaps' to enable it to kick it off.
The specifics of it would be a bit tricky for some things, but on a basic level I don't think it'd be particularly difficult.
That said, the other way would be to make use of the export to collada feature.
Build out your level, select relevent features, export to collada. Then import into your program of choice, bake out the individual object's tonemaps via script or something similar. 'd probably take a few days to write out a script to run all that(or maybe just parse the mission and load the associated assets if you wanted to get really clever) in Blender.
So yeah, there's a few options, but I think the 'best' would be the first one, where you have a light-weight lightmap system built in than trying to jump between programs to get the same work done.
(While we're thinking, doing additional stuff like light probes for simulated ambient lighting from the tonemap would be a pretty nice bonus to this setup as well)
That said, I've never personally done tone map baking, so I'm just running on theory here. Any chance you could write a mini tutorial on how you got the above results Bryce? At minimum, it'd make for a good blog or resource or whatnot.
04/10/2013 (7:16 am)
I'm willing to bet that you could actually probably do it (somewhat) easily using some really high detail shadowmaps, and some other clever tricks to simulate light bouncing and then bake it down using the model's UV's and save them all to a subfolder for the mission to use.From there, you could probably do a dynamic hijack of materials for certain objects on-load and re purpose the tonemap field to use the level-generated ones.
Just add a field to statics like 'useLightmaps' to enable it to kick it off.
The specifics of it would be a bit tricky for some things, but on a basic level I don't think it'd be particularly difficult.
That said, the other way would be to make use of the export to collada feature.
Build out your level, select relevent features, export to collada. Then import into your program of choice, bake out the individual object's tonemaps via script or something similar. 'd probably take a few days to write out a script to run all that(or maybe just parse the mission and load the associated assets if you wanted to get really clever) in Blender.
So yeah, there's a few options, but I think the 'best' would be the first one, where you have a light-weight lightmap system built in than trying to jump between programs to get the same work done.
(While we're thinking, doing additional stuff like light probes for simulated ambient lighting from the tonemap would be a pretty nice bonus to this setup as well)
That said, I've never personally done tone map baking, so I'm just running on theory here. Any chance you could write a mini tutorial on how you got the above results Bryce? At minimum, it'd make for a good blog or resource or whatnot.
#20
04/10/2013 (8:04 am)
Torque generates lightmaps for the terrain (the "lighting mission" phase) so perhaps this could be extended to add lightmapping to objects, too.
Torque Owner Richard Ranft
Roostertail Games
Not really helpful, I know - but maybe it's a starting point?