Sahara for Torque 3D MIT 3.5.1...
by Quinton Delpeche · in Torque 3D Professional · 07/11/2014 (5:24 am) · 43 replies
I am pleased to announce that Konrad Kiss has decided to allow me to release this to the Torque 3D MIT community via a pull request. He has stated that any licensing questions should be directed at him via @KonradKiss.
I am busy doing the final integration testing now and I have a last few steps to check before I am ready to do the pull request. But before I get ahead of myself, let me just give you all some further information.
You can get more information from the BitGap Games website: www.bitgap.com/products/12/sahara.html
So where are we right now? Well I have all the engine changes merged in and I have a test build running, I am currently doing my last tests to make sure that it is stable ... luckily Konrad's code is top notch and there haven't been any gotchas.
What can you do? Well my Git knowledge is a little fuzzy ... almost non-existent ... so I need some help on doing the Pull request for this code. So if anyone can assist me here, I will get the Pull request done and then do a new blog with some updates.
There are also two extra source files that need to be added "accuFeature.cpp" and "accuFeature.h", currently these reside in a folder called "sahara" in the root of the "Engine/source/" folder. Where ideally should these files be placed and/or renamed to better fit in with the current coding standards.
So if anyone can help me getting this Pull request done ... and feel free to send Konrad some thank yous ... this is very generous of him.
Edit:
I have created a GitHub repository for this and I am waiting for the Steering Committee to approve the Pull Request. The repository can be found here.
I have also created a resource for the time being, read it here.
I am busy doing the final integration testing now and I have a last few steps to check before I am ready to do the pull request. But before I get ahead of myself, let me just give you all some further information.
Quote:Sahara is a tech extension to the Torque 3D 1.1 game engine ( 1.1 Beta 2 and later versions) by GarageGames which lets you create more realistic game environments by generating naturally accumulated dirt, snow, dust or any other material over your game shapes. It is a successor to our Cliff Construction Kit.
The motto of Sahara is: use one model in many environments. Sahara can be set up on a per material basis. Once installed, use the Torque 3D Material Editor to edit the properties of your shape.
You can get more information from the BitGap Games website: www.bitgap.com/products/12/sahara.html
So where are we right now? Well I have all the engine changes merged in and I have a test build running, I am currently doing my last tests to make sure that it is stable ... luckily Konrad's code is top notch and there haven't been any gotchas.
What can you do? Well my Git knowledge is a little fuzzy ... almost non-existent ... so I need some help on doing the Pull request for this code. So if anyone can assist me here, I will get the Pull request done and then do a new blog with some updates.
There are also two extra source files that need to be added "accuFeature.cpp" and "accuFeature.h", currently these reside in a folder called "sahara" in the root of the "Engine/source/" folder. Where ideally should these files be placed and/or renamed to better fit in with the current coding standards.
So if anyone can help me getting this Pull request done ... and feel free to send Konrad some thank yous ... this is very generous of him.
Edit:
I have created a GitHub repository for this and I am waiting for the Steering Committee to approve the Pull Request. The repository can be found here.
I have also created a resource for the time being, read it here.
About the author
Gobbo Games is an Independent Games Development company operating from Durban in South Africa. We believe in creating high-quality cost-effective games that remain true to the belief of Independent Game Developers around the world.
#22
All the accumulation settings are set to max for debugging purposes because I wanted to be sure it was working. The material of the left box is the green test material, the right box is the orange test material. Neither objects have an accumulation map assigned, the handles for the accumulation map shader point to the material used by the accumulation volumes. I just need to separate individual shape instances like those two barrels and then make the volumes automatically recalculate at the right times and then I'll get it up for testing.
07/11/2014 (6:11 pm)
Not quite ready for public consumption yet but I've got the prototype working:All the accumulation settings are set to max for debugging purposes because I wanted to be sure it was working. The material of the left box is the green test material, the right box is the orange test material. Neither objects have an accumulation map assigned, the handles for the accumulation map shader point to the material used by the accumulation volumes. I just need to separate individual shape instances like those two barrels and then make the volumes automatically recalculate at the right times and then I'll get it up for testing.
#23
Massive thanks for your addition to the engine
And a big thanks to Q ( ;) ) and Andrew for spearheading getting this integrated.
One thought occurs if we're moving towards flexibility, Andrew:
Would it be possible to have the accumulation volumes interact with a screen-space shader?
So it'd get the pixels inside the volume, determine if the normal angle is valid and then blend the materials together?
That would mean you could even localize the accumulation effect, such as a broken skylight allowing snow/rain to accumulate in the exposed area.
If I recall correctly, the Cryengine does something similar to that for it's rain effets by having a special type of light(since lights grab the pixels inside their volumes as-is) and just applying the material in question overtop the regular surface. I'd think we could do something similar for this.
07/11/2014 (8:19 pm)
@KonradMassive thanks for your addition to the engine
And a big thanks to Q ( ;) ) and Andrew for spearheading getting this integrated.
One thought occurs if we're moving towards flexibility, Andrew:
Would it be possible to have the accumulation volumes interact with a screen-space shader?
So it'd get the pixels inside the volume, determine if the normal angle is valid and then blend the materials together?
That would mean you could even localize the accumulation effect, such as a broken skylight allowing snow/rain to accumulate in the exposed area.
If I recall correctly, the Cryengine does something similar to that for it's rain effets by having a special type of light(since lights grab the pixels inside their volumes as-is) and just applying the material in question overtop the regular surface. I'd think we could do something similar for this.
#24
With a little bit of tweaking, you can get some pretty decent results.
This is going to really help with Re-Spawn as I have similar objects being used in maps in different biomes and all I need to do is apply the accumulator. I have set up my objects to use skins, so my materials look something like this.
So now I just place the objects ... set the skin and I am good to go. The same will apply to the natural objects like rocks and stuff ... I am very happy that I was given the opportunity to share this with everyone.
Kudos to Konrad for letting me. 8-}
07/12/2014 (1:12 am)
So here are some test images that I have been working. THis were quickly created in Medusa Pro, placed in the World Editor and then had the accumulation placed on them.With a little bit of tweaking, you can get some pretty decent results.
This is going to really help with Re-Spawn as I have similar objects being used in maps in different biomes and all I need to do is apply the accumulator. I have set up my objects to use skins, so my materials look something like this.
singleton Material( MAT_Base_PDiamondPlate )
{
mapTo = "Base_PDiamondPlate";
diffuseMap[0] = "./textures/Base_PDiamondPlate_D.dds";
normalMap[0] = "./textures/Base_PDiamondPlate_N.dds";
specularMap[0] = "./textures/Base_PDiamondPlate_S.dds";
specular[0] = "1 1 1 1";
specularPower[0] = "10";
translucentBlendOp = "None";
useAnisotropic[0] = "1";
materialTag0 = "Platforms";
};singleton Material( MAT_Sand_PDiamondPlate )
{
mapTo = "Sand_PDiamondPlate";
diffuseMap[0] = "./textures/Base_PDiamondPlate_D.dds";
normalMap[0] = "./textures/Base_PDiamondPlate_N.dds";
specularMap[0] = "./textures/Base_PDiamondPlate_S.dds";
specular[0] = "1 1 1 1";
specularPower[0] = "10";
translucentBlendOp = "None";
useAnisotropic[0] = "1";
accuMap[0] = "art/terrains/DesertA/sand05.dds";
accuStrength[0] = "0.5";
accuCoverage[0] = "1.8";
accuSpecular[0] = "0.5";
materialTag0 = "Platforms";
};singleton Material( MAT_Snow_PDiamondPlate )
{
mapTo = "Snow_PDiamondPlate";
diffuseMap[0] = "./textures/Base_PDiamondPlate_D.dds";
normalMap[0] = "./textures/Base_PDiamondPlate_N.dds";
specularMap[0] = "./textures/Base_PDiamondPlate_S.dds";
specular[0] = "1 1 1 1";
specularPower[0] = "10";
translucentBlendOp = "None";
useAnisotropic[0] = "1";
accuMap[0] = "art/terrains/ArcticA/snow02.dds";
accuStrength[0] = "0.5";
accuCoverage[0] = "1.8";
accuSpecular[0] = "0.5";
materialTag0 = "Platforms";
};So now I just place the objects ... set the skin and I am good to go. The same will apply to the natural objects like rocks and stuff ... I am very happy that I was given the opportunity to share this with everyone.
Kudos to Konrad for letting me. 8-}
#25
Those shots look great, Q!
Cool idea, Jeff. That'd be an interesting way to go about doing it. Though I imagine decals might produce more natural-looking results? Maybe not for rain, but for anything like snow which might drift.
07/12/2014 (2:54 am)
Andrew - awesome! Do you reckon this is something we could shoehorn into 3.6? It'd be nice to have some additional eye-candy to put in the release notes ;).Those shots look great, Q!
Cool idea, Jeff. That'd be an interesting way to go about doing it. Though I imagine decals might produce more natural-looking results? Maybe not for rain, but for anything like snow which might drift.
#26
I don't want to say yes on 3.6 only to have myself run straight into a brick wall and then fall off track like I did with Assimp. With that said, I think I may have this wrapped up real soon.
Here's the current state of affairs:
The problem I'm having is also showcased in the video. It seems as though textures are only setup once per-material, per-frame rather than per-object so the accumulation map of the first object in view applies to all the objects in view. So, as is, if you had your biomes on different sides of the map, it would work fine. I'm not satisfied with that though and will keep on digging until I find a solution.
07/12/2014 (11:19 am)
@ DanI don't want to say yes on 3.6 only to have myself run straight into a brick wall and then fall off track like I did with Assimp. With that said, I think I may have this wrapped up real soon.
Here's the current state of affairs:
The problem I'm having is also showcased in the video. It seems as though textures are only setup once per-material, per-frame rather than per-object so the accumulation map of the first object in view applies to all the objects in view. So, as is, if you had your biomes on different sides of the map, it would work fine. I'm not satisfied with that though and will keep on digging until I find a solution.
#27
It's been a few months / years since I worked with Torque, but I think that to make this work, the accumulation feature should be taken out of the material parts and become a separate thing to not have to deal with cloning material instances on the fly (which used to be the way to have unique textures on a shape's instance as far as I remember). Instead, the shape could contain a reference to an AccumulationVolume that describes the feature. When you need to pass accumulation data to the pixel shader, you should be able to access the shape instance - without having to use the material as a source. (Please let me know if I'm way off with this.)
It might be worthwhile to plan ahead and find a way for the user to be able to set accumulation property overrides on a per material basis since, for example cold and hot surfaces will accumulate snow differently even within the same volume (assuming different textures of course). The UI could quickly get confusing here, and I'm not sure what the best way to do this would be. It is possible that keeping a single override on/off switch on the material would do a great job, and the rest we'd just have to hire artists for. :)
07/12/2014 (12:04 pm)
@Andrew: Right, the setup was made that way so that a single model with several materials could have different accumulation textures for each of the mats. Say, for a bridge with snow on top and ice on the bottom. But these corner cases never really surfaced, because most such objects had a single material for the geometry that could have been affected (that is if they were optimized properly).It's been a few months / years since I worked with Torque, but I think that to make this work, the accumulation feature should be taken out of the material parts and become a separate thing to not have to deal with cloning material instances on the fly (which used to be the way to have unique textures on a shape's instance as far as I remember). Instead, the shape could contain a reference to an AccumulationVolume that describes the feature. When you need to pass accumulation data to the pixel shader, you should be able to access the shape instance - without having to use the material as a source. (Please let me know if I'm way off with this.)
It might be worthwhile to plan ahead and find a way for the user to be able to set accumulation property overrides on a per material basis since, for example cold and hot surfaces will accumulate snow differently even within the same volume (assuming different textures of course). The UI could quickly get confusing here, and I'm not sure what the best way to do this would be. It is possible that keeping a single override on/off switch on the material would do a great job, and the rest we'd just have to hire artists for. :)
#28
07/13/2014 (1:18 pm)
This stuff blows my mind. I'm not familiar with accumulation. Does this allow you to set textures for different 'seasons' which can fade in and out based on a time value? What I'm asking is how are the transitions handled, using some global variable or off of the TimeofDay variables? 
#29
07/13/2014 (2:29 pm)
Jesse - I think you could do that, I've seen examples where Sahara was used to show snow gradually accumulating on objects by animating the strength of the accumulation.
#30
www.youtube.com/watch?v=9-zQihAyfzo
07/13/2014 (2:43 pm)
Yep, there was a resource to do it, here is the effect:www.youtube.com/watch?v=9-zQihAyfzo
#31
I've been perusing the boards a bit, and at every turn I'm met with these astounding improvements that are all in the works: deferred shading, occlusion culling(man please let this hit 3.6 or so..), dx refactor... If I was just a tad bit better with merging the source (honestly, as Richard would put it, "if I get my head out my ass" I could probably figure it out) the combination of all of these features could be very powerful. Just wish there was a pre-compiled version of all of this stuff for the more artistically-inclined, such as myself =)
edit: I just checked AndrewMac's thread about the occlusion culling, and it appears that wasn't entirely completed and I don't believe that was intended for 3.6 my mistake. That's a bit unfortunate, on the level that I am working on here it seems fairly obvious the occlusion culling in T3D is lacking.
07/13/2014 (3:04 pm)
Wow. I can't believe how under-rated Torque engine seems to be. I mean, the batteries aren't included but ultimately no matter what you use you are going to need a little custom scripting. I've been perusing the boards a bit, and at every turn I'm met with these astounding improvements that are all in the works: deferred shading, occlusion culling(man please let this hit 3.6 or so..), dx refactor... If I was just a tad bit better with merging the source (honestly, as Richard would put it, "if I get my head out my ass" I could probably figure it out) the combination of all of these features could be very powerful. Just wish there was a pre-compiled version of all of this stuff for the more artistically-inclined, such as myself =)
edit: I just checked AndrewMac's thread about the occlusion culling, and it appears that wasn't entirely completed and I don't believe that was intended for 3.6 my mistake. That's a bit unfortunate, on the level that I am working on here it seems fairly obvious the occlusion culling in T3D is lacking.
#32
The trick was following a similar path as dynamic cubemaps:
github.com/GarageGames/Torque3D/blob/8142a3e9645e881cd8e21434809289855e57a0b4/En...
I'll clean the code up and get a repo up for testing.
07/14/2014 (5:58 pm)
Bam! Got it.The trick was following a similar path as dynamic cubemaps:
github.com/GarageGames/Torque3D/blob/8142a3e9645e881cd8e21434809289855e57a0b4/En...
I'll clean the code up and get a repo up for testing.
#33
07/14/2014 (10:31 pm)
Excellent! That is very cool.... 
#34
Accumulation Volumes ... that is a very smart idea. Nice work Mr Mac. :)
07/15/2014 (6:14 am)
Quote:
Excellent! That is very cool....
Accumulation Volumes ... that is a very smart idea. Nice work Mr Mac. :)
#35
07/15/2014 (11:26 am)
@AndrewMac: Nice, man. I can't thank you enough for spearheading the charge forward with Torque3D.
#36
07/15/2014 (11:28 am)
Fantastic work, Andrew!
#37
First version ready for testing. It's pretty simple to use. Add an accumulation volume (found under Level in the editor), set the texture to whatever accumulation texture you want. Next, import your object and enable accumulation in the material editor. When you move the object inside the volume the object will have access to the texture you chose, so you can tweak the accumulation settings just like you normally would with Sahara.
Right now it only supports TSStatics but there's no real limitation there. The system works by registering an object with the accumulation volumes if it has accumulation on in the material. Then you use an update command when the object changes position. That recalculates which, if any, accumulation volumes the object is in. Technically you could add this to shapebase and accumulation would turn on when the player moves inside the volume. Should note right now it checks position not bounding box.
My hope is that with static objects and a single accumulation volume in view it should be no different in speed than the original Sahara. The code is still a little rough though so who knows, it may be slower.
07/15/2014 (10:27 pm)
github.com/andr3wmac/Torque3D/tree/accuVolumesFirst version ready for testing. It's pretty simple to use. Add an accumulation volume (found under Level in the editor), set the texture to whatever accumulation texture you want. Next, import your object and enable accumulation in the material editor. When you move the object inside the volume the object will have access to the texture you chose, so you can tweak the accumulation settings just like you normally would with Sahara.
Right now it only supports TSStatics but there's no real limitation there. The system works by registering an object with the accumulation volumes if it has accumulation on in the material. Then you use an update command when the object changes position. That recalculates which, if any, accumulation volumes the object is in. Technically you could add this to shapebase and accumulation would turn on when the player moves inside the volume. Should note right now it checks position not bounding box.
My hope is that with static objects and a single accumulation volume in view it should be no different in speed than the original Sahara. The code is still a little rough though so who knows, it may be slower.
#38
@Andrew: Awesome job man ... gosh it makes me so happy to see stuff like this happen. 8-}
07/15/2014 (10:48 pm)
And thats why OpenSource works!!!@Andrew: Awesome job man ... gosh it makes me so happy to see stuff like this happen. 8-}
#39
07/15/2014 (11:35 pm)
i.imgur.com/2AlQVag.jpg Clarify intent behind direction? Other than that, sweet extension there. 
Chris DeBoy