Enviro-Torque 2.0 Update: New Cloud-System Resource
by Steven Peterson · 05/24/2006 (10:26 am) · 24 comments
Enviro-Torque 2.0 Update: New Cloud-System Resource
Enviro-Torque 2.0 is progressing along slow and steady. The biggest challenge so far has been fixing the way Torque handles cloud-layers. To gain the functionality I want, I've had to entirely redesign this system, and am hoping to release it as a free-resource soon!
This is a 'brief' overview of how the new system works, the problems solved, and where I still need some help. Feedback is encouraged!
New Torque Cloud-System Resource:
1. Design Goals and Intended Features:
- Control individual cloud layers (on/off).
- Ability to fade layer alpha to any value between 0 and 1.
- Support for arbitrary number of cloud layers.
- Allow for easy addition of new procedural effects for generating cloud layers.
- Allow for different types of textured and procedural cloud layers to be used simultaneously.
- Specifically include:
- - Basic textured layers.
- - Rob Rendell's Perlin based cloud generator.
- - Joseph Jahn's fractal cloud generator.
2. Why the old Cloud-System didn't work:
2.1 The Sky Class:
A look at the old code shows that the Sky class's main purpose is to draw and manage the skybox including effects like visible distance, fog-bands and sky-textures. In my view this is the only thing it should be doing.
In reality, it also manages wind, volumetric fog and cloud layers just to name a few. While these effects are tightly integrated in their current implementation, most of them should be their own classes with low or no coupling between them.
2.2 The Cloud Class:
The Cloud class, for reasons unknown, can also be found in the sky.h/cc files. Even in the old system each cloud layer was it's own object-instance. Regardless, many properties, for example, if the layer was on or off, were properties of the Sky class not the individual cloud-layers.
Making the fundamental design changes I required just did not seem possible, so I looked for other avenues.
3. A New Approach:
3.1 A Clean Slate and fxCloudLayer:
First I stripped all the cloud stuff out of sky.h/cc. I momentarily had the old system working with the Cloud class located in cloud.h/cc to prove it worked; then I dumped it. From now on my work would be in /engine/game/clouds/.
Second I got in touch with Rob Rendell who had started an fxCloudLayer class. The idea was that you would add cloud-layers similar to fxSunLight objects. fxCloudLayer was essentially a single basic cloud-layer with an optional Perlin-fractal generator for random cloud creation.
One of my design-goals was easy addition of new procedural effects for generating cloud layers. So, dropping the 'fx' and taking out the procedural-code, I created an abstract base class: CloudLayer. Through inheritance I quickly came up with concrete classes:
- BasicCloudLayer - texture from image.
- RendellCloudLayer - Rendell's Perlin implementation.
- JahnCloudLayer - Jahn's plasma-fractal implementation.
So far everything worked great.
3.2 UpdateSpeed() - A Bug Fix:
In current versions of TGE cloud-layers have a 'cloud-speed' which gets multiplied by the wind-velocity to adjust the rate each cloud texture scrolls across the sky. Unfortunately it only updates itself when you hit 'Apply'.
I added an updateSpeed() function that gets called from CloudLayer's renderObject() which checks the wind-velocity every frame. It would probably be more efficient to have the wind-velocity notify the cloud-layers when there was a change, but that's another day.
Note: This getWindVelocity() call is the only way in which my system is coupled to the Sky class!
3.3 UpdateAlpha() A New Feature:
To date I don't understand how this was done in the past.
I rewrote it so that: each layer has it's own mTargetAlpha and mTargetTime. renderObject() also calls updateAlpha(). Here, I get the difference between the layer's current alpha-level and mTargetAlpha. Using that info, the last 'change in time', the time remaining and a touch of calculus, I interpolate the current alpha level towards the target. The final equation can be seen below:
The advantage to this approach is that on any given frame all I care about is:
- how far do I have to go (difference in alpha)
- how long do i have to get there (difference in time)
- ( and what was the last change in time if-applicable )
I don't care if the clouds are fadeing in/out, so I can change mTargetAlpha and/or mTargetTime at any point, even mid-fade - no problem! mTargetAlpha can also be any alpha level, not just 0 or 1. And finally - each layer does this independently.
Fig. 3. A quick UML diagram.
4. CloudManager:
I wanted a common interface from which you could control all the cloud-layers in-simiulation, wheather using the console, script or C++ code. The answer was singleton-simObject CloudManager.
As part of their onAdd function, inherrited from CloudLayer, all cloud layers 'register' with CloudManager by calling CloudManager::registerLayer( this ). The handle then gets stored in a vector. Now we have reliable handles to all our cloud-layers in one place.
Right now CloudManager only provides a few simple function calls:
Up to this point everything (with a single exception) works great! With just a touch of inherritance, the solution is elegant, organized, and flexible and I have hit all my design goals (save one). This is where the whole thing gets ugly and complicated and if you see a better way PLEASE LET ME KNOW!!!.
5. The Last Wall of the Castle: Multi-Layered Procedural Effects
Untill this point JahnCloudLayer works functionally, as an object, but not visually. Written for the old cloud-system it is not a single cloud-layer, but a composit of three cloud layers to create a single visual effect.
I tried using additive-blending to merge the 3-layers into one but for some reason, it isn't even close to the same. There may be a different way to 'blend', 'merge', or 'layer' three bitmaps into one but this is well beyond my realm of knowlege. A closer examination reveils that even if blending worked the three layers don't actually tile! This is normally hidden when the three layers are moving at different rates in real-time.
How best to solve the problem?
Fig. 4. Jahn Fractal Clouds as it's supposed to look:
Fig. 5. Jahn Fractal Clouds as they look now with additive blending:
6. SubLayers
6.1 Overview
Fig. 6. Another quick UML diagram
The next approach I tried was for each CloudLayer to be a composite of one or more sublayers. So I created an abstract class SubLayer which takes on most of the former reponsibility of CloudLayer. CloudLayer gets an array of SubLayers to control.
At this point CloudLayer is still the (scene)object that gets added to your *.mis. However it's the SubLayer(s) that are the sceneobjects that get rendered. CloudLayer initially sets up the texture in setTexture() and then calls setTexture( textureHandle ) on each of the subObjects. CloudLayer is also still the one registering with CloudManager. It is mostly acting as an interface class.
6.2 Problems with this Approach
In execution, this approach is only a handful of extra lines. For the programmer, the complexity doubles with the addition of sublayers. Because there is now a SubLayer, BasicSubLayer, JahnSubLayer, etc. in addition to CloudLayer, BasicCloudLayer, JahnCloudLayer, etc. it is litterly twice as many files. It seems my solution went from 'elegant' to 'explosion in a paper-mill' overnight.
So far I haven't gotten this working yet. Today's problem is that while CloudLayers are the objects added to the *.mis and thereby added to the simulation correctly; SubLayers are instanciated from C++ and so I have to register them myself. Based on the 'Adding already added object' errors, I gather I'm doing this wrong.
I'm calling registerObject() in the SubLayer child-objects' constructors. That seems to ensure onAdd() (inherited from SubLayer) gets called. I'm still not sure how this is going wrong but it is...
7. Final Conclusion:
I think this is one of Torque's systems that was in dire need of re-engineering. It will definitely be a huge milestone in the completion of EnviroTorque 2.0 and I hope it will prove a huge leap forward for the Torque engine as a whole.
I think the idea of SubLayers is a huge mis-step in an otherwise solid design. However, getting the Jahn Fractal Clouds properly integrated to work alongside static-textured cloud layers is a key feature. At this point I'm open to ideas...
Last Idea:
I suppose it would be possible for just the JahnCloudLayer to maintain an array of three texture-handles and I could probably adjust properties like cloud-speed for each of them. Then I could just layer the three layers together into one texture in RenderObject(). This last step I'm not sure how to do though.
8. Help Wanted:
Torque Networking is something I just don't understand yet. I'm looking for a Torque-Veteran who will be able to properly finish off the networking aspect of this resource. Essentially netmasks, packUpdate(), and unPackUpdate() functions.
I'm also looking for a small handful of Torque-Veterans to beta-test it before I make it a final resource. Testing will happen as soon as I have a release-candidate to email out.
If your interested in helping please find the email-address in my profile or the contact box on my website to let me know!
And lastly - Thanks for reading!
Enviro-Torque 2.0 is progressing along slow and steady. The biggest challenge so far has been fixing the way Torque handles cloud-layers. To gain the functionality I want, I've had to entirely redesign this system, and am hoping to release it as a free-resource soon!
This is a 'brief' overview of how the new system works, the problems solved, and where I still need some help. Feedback is encouraged!
New Torque Cloud-System Resource:1. Design Goals and Intended Features:
- Control individual cloud layers (on/off).
- Ability to fade layer alpha to any value between 0 and 1.
- Support for arbitrary number of cloud layers.
- Allow for easy addition of new procedural effects for generating cloud layers.
- Allow for different types of textured and procedural cloud layers to be used simultaneously.
- Specifically include:
- - Basic textured layers.
- - Rob Rendell's Perlin based cloud generator.
- - Joseph Jahn's fractal cloud generator.
2. Why the old Cloud-System didn't work:
2.1 The Sky Class:
A look at the old code shows that the Sky class's main purpose is to draw and manage the skybox including effects like visible distance, fog-bands and sky-textures. In my view this is the only thing it should be doing.
In reality, it also manages wind, volumetric fog and cloud layers just to name a few. While these effects are tightly integrated in their current implementation, most of them should be their own classes with low or no coupling between them.
2.2 The Cloud Class:
The Cloud class, for reasons unknown, can also be found in the sky.h/cc files. Even in the old system each cloud layer was it's own object-instance. Regardless, many properties, for example, if the layer was on or off, were properties of the Sky class not the individual cloud-layers.
Making the fundamental design changes I required just did not seem possible, so I looked for other avenues.
3. A New Approach:
3.1 A Clean Slate and fxCloudLayer:
First I stripped all the cloud stuff out of sky.h/cc. I momentarily had the old system working with the Cloud class located in cloud.h/cc to prove it worked; then I dumped it. From now on my work would be in /engine/game/clouds/.
Second I got in touch with Rob Rendell who had started an fxCloudLayer class. The idea was that you would add cloud-layers similar to fxSunLight objects. fxCloudLayer was essentially a single basic cloud-layer with an optional Perlin-fractal generator for random cloud creation.
One of my design-goals was easy addition of new procedural effects for generating cloud layers. So, dropping the 'fx' and taking out the procedural-code, I created an abstract base class: CloudLayer. Through inheritance I quickly came up with concrete classes:
- BasicCloudLayer - texture from image.
- RendellCloudLayer - Rendell's Perlin implementation.
- JahnCloudLayer - Jahn's plasma-fractal implementation.
So far everything worked great.
3.2 UpdateSpeed() - A Bug Fix:
In current versions of TGE cloud-layers have a 'cloud-speed' which gets multiplied by the wind-velocity to adjust the rate each cloud texture scrolls across the sky. Unfortunately it only updates itself when you hit 'Apply'.
I added an updateSpeed() function that gets called from CloudLayer's renderObject() which checks the wind-velocity every frame. It would probably be more efficient to have the wind-velocity notify the cloud-layers when there was a change, but that's another day.
Note: This getWindVelocity() call is the only way in which my system is coupled to the Sky class!
3.3 UpdateAlpha() A New Feature:
To date I don't understand how this was done in the past.
I rewrote it so that: each layer has it's own mTargetAlpha and mTargetTime. renderObject() also calls updateAlpha(). Here, I get the difference between the layer's current alpha-level and mTargetAlpha. Using that info, the last 'change in time', the time remaining and a touch of calculus, I interpolate the current alpha level towards the target. The final equation can be seen below:
The advantage to this approach is that on any given frame all I care about is:
- how far do I have to go (difference in alpha)
- how long do i have to get there (difference in time)
- ( and what was the last change in time if-applicable )
I don't care if the clouds are fadeing in/out, so I can change mTargetAlpha and/or mTargetTime at any point, even mid-fade - no problem! mTargetAlpha can also be any alpha level, not just 0 or 1. And finally - each layer does this independently.
stormAlpha = currentStormAlpha + ( alphaToGo / ( ( timeToGo + dT ) / dT ) ); [i]where:[/i] stormAlpha = new layer alpha CurrentStormAlpha = current layer alpha dT (delta-Time) = change in time
Fig. 3. A quick UML diagram.
4. CloudManager:
I wanted a common interface from which you could control all the cloud-layers in-simiulation, wheather using the console, script or C++ code. The answer was singleton-simObject CloudManager.
As part of their onAdd function, inherrited from CloudLayer, all cloud layers 'register' with CloudManager by calling CloudManager::registerLayer( this ). The handle then gets stored in a vector. Now we have reliable handles to all our cloud-layers in one place.
Right now CloudManager only provides a few simple function calls:
void layerToggle( U32 layer, SimTime time ); /// toggles layer on/off void layerToggleAll( SimTime time ); /// toggles all cloud layers void layerSet( U32 layer, F32 alpha, SimTime time ); void layerSetAll( F32 alpha, SimTime time );
Up to this point everything (with a single exception) works great! With just a touch of inherritance, the solution is elegant, organized, and flexible and I have hit all my design goals (save one). This is where the whole thing gets ugly and complicated and if you see a better way PLEASE LET ME KNOW!!!.
5. The Last Wall of the Castle: Multi-Layered Procedural Effects
Untill this point JahnCloudLayer works functionally, as an object, but not visually. Written for the old cloud-system it is not a single cloud-layer, but a composit of three cloud layers to create a single visual effect.
I tried using additive-blending to merge the 3-layers into one but for some reason, it isn't even close to the same. There may be a different way to 'blend', 'merge', or 'layer' three bitmaps into one but this is well beyond my realm of knowlege. A closer examination reveils that even if blending worked the three layers don't actually tile! This is normally hidden when the three layers are moving at different rates in real-time.
How best to solve the problem?
Fig. 4. Jahn Fractal Clouds as it's supposed to look:
Fig. 5. Jahn Fractal Clouds as they look now with additive blending:
6. SubLayers
6.1 Overview
Fig. 6. Another quick UML diagram
The next approach I tried was for each CloudLayer to be a composite of one or more sublayers. So I created an abstract class SubLayer which takes on most of the former reponsibility of CloudLayer. CloudLayer gets an array of SubLayers to control.
At this point CloudLayer is still the (scene)object that gets added to your *.mis. However it's the SubLayer(s) that are the sceneobjects that get rendered. CloudLayer initially sets up the texture in setTexture() and then calls setTexture( textureHandle ) on each of the subObjects. CloudLayer is also still the one registering with CloudManager. It is mostly acting as an interface class.
6.2 Problems with this Approach
In execution, this approach is only a handful of extra lines. For the programmer, the complexity doubles with the addition of sublayers. Because there is now a SubLayer, BasicSubLayer, JahnSubLayer, etc. in addition to CloudLayer, BasicCloudLayer, JahnCloudLayer, etc. it is litterly twice as many files. It seems my solution went from 'elegant' to 'explosion in a paper-mill' overnight.
So far I haven't gotten this working yet. Today's problem is that while CloudLayers are the objects added to the *.mis and thereby added to the simulation correctly; SubLayers are instanciated from C++ and so I have to register them myself. Based on the 'Adding already added object' errors, I gather I'm doing this wrong.
I'm calling registerObject() in the SubLayer child-objects' constructors. That seems to ensure onAdd() (inherited from SubLayer) gets called. I'm still not sure how this is going wrong but it is...
7. Final Conclusion:
I think this is one of Torque's systems that was in dire need of re-engineering. It will definitely be a huge milestone in the completion of EnviroTorque 2.0 and I hope it will prove a huge leap forward for the Torque engine as a whole.
I think the idea of SubLayers is a huge mis-step in an otherwise solid design. However, getting the Jahn Fractal Clouds properly integrated to work alongside static-textured cloud layers is a key feature. At this point I'm open to ideas...
Last Idea:
I suppose it would be possible for just the JahnCloudLayer to maintain an array of three texture-handles and I could probably adjust properties like cloud-speed for each of them. Then I could just layer the three layers together into one texture in RenderObject(). This last step I'm not sure how to do though.
8. Help Wanted:
Torque Networking is something I just don't understand yet. I'm looking for a Torque-Veteran who will be able to properly finish off the networking aspect of this resource. Essentially netmasks, packUpdate(), and unPackUpdate() functions.
I'm also looking for a small handful of Torque-Veterans to beta-test it before I make it a final resource. Testing will happen as soon as I have a release-candidate to email out.
If your interested in helping please find the email-address in my profile or the contact box on my website to let me know!
And lastly - Thanks for reading!
#2
The layout is also a lot more cluttered than I wanted. Live and learn I guess :'-(
[edit]Made another pass, it should be all ok by now![/edit]
05/24/2006 (10:56 am)
Sorry for the technical difficulties everyone! The website initially choked everywhere I used a quotation-mark or a hyphen and there's ALOT of them...The layout is also a lot more cluttered than I wanted. Live and learn I guess :'-(
[edit]Made another pass, it should be all ok by now![/edit]
#3
05/24/2006 (10:57 am)
Wow! Nice work! 
#4
05/24/2006 (11:48 am)
Well I'm floored. This will definitely be a huge leap forward for the engine, no hoping necessary. Wish I knew enough to help out, although I could certainly beta-test it. Great work so far. 
#5
05/24/2006 (12:32 pm)
I love your diagrams and blog-design, keep up the good work! :) 
#7
- Peterk
05/24/2006 (1:06 pm)
When we spoke on irc you had a great attitude. In addition to your willingness to learn from those around you, you display an ability to get things done. Excellent work, I hope your future is filled with success in this industry.- Peterk
#9
05/24/2006 (2:58 pm)
Nice work, Steven... I'll be in touch. :)
#10
05/24/2006 (2:59 pm)
This is looking great!
#11
05/24/2006 (3:06 pm)
Just an little question? What did you use for the UML diagrams? :)
#12
Thats ok probably for clouds (never culled), but imagine an animating thing where you look away from it and it stops animating? that could look strange.
Just FYI.
05/24/2006 (3:11 pm)
Hmm, just reading what you posted, you probably dont want to do your updates during renderObject, I guess it might be ok in this case, but generally you'd want to do those in processTick. The reason I say its not a good idea to do it in renderobject is that sometimes objects get culled out when not in view.Thats ok probably for clouds (never culled), but imagine an animating thing where you look away from it and it stops animating? that could look strange.
Just FYI.
#13
@Ben - Me too. :-)
@Phil - The UML program was UML Sculptor.
@Phil - Thanks for the great tip on processTick(). Since they are sceneObjects, iTickable() may be more helpfull, as-per our IRC convo. I'll check them both out though!
05/25/2006 (7:36 am)
Thanks everyone for the great comments!@Ben - Me too. :-)
@Phil - The UML program was UML Sculptor.
@Phil - Thanks for the great tip on processTick(). Since they are sceneObjects, iTickable() may be more helpfull, as-per our IRC convo. I'll check them both out though!
#14
06/01/2006 (11:58 am)
Wow I can't wait! This is really cool.
#15
06/01/2006 (9:12 pm)
I could add stars at night if you havn't already done that
#16
Personally, I'm also intending to use higher-up layers to portray "atmospheric haze" when lower, denser cloud-layers are not present.
The combinations are near endless...
06/07/2006 (9:29 pm)
Since there are no restrictions on number or type of cloud-layers, creating a star-map layer would be easy. Additionally you could now fade the stars in/out at dusk/dawn (respectively). The unsolved problem with this layer is the stars may appear in-front of the moon depending on your implementation of stars & moon. We're definitely 1-step closer though. Personally, I'm also intending to use higher-up layers to portray "atmospheric haze" when lower, denser cloud-layers are not present.
The combinations are near endless...
#17
On an offtopic note, I leafed through your website a bit and I love your idea for Zelda. I had a similar idea for a graphic update to goldeneye. Its an N64 game yes but its a classic love of mine none the less.
06/08/2006 (7:52 pm)
Ahhhh, the only thing I hate about previews is waiting for the release. Very sweet stuff. One question is could the skycolor be controled by time fading between a selection of day/night/sunrise-set colors? If it could, combined with the limitless layers I really dont think the engine would need anything else for the skys realism.On an offtopic note, I leafed through your website a bit and I love your idea for Zelda. I had a similar idea for a graphic update to goldeneye. Its an N64 game yes but its a classic love of mine none the less.
#18
I completely agree on previews and delays so I apologize for this one. I wanted to be done by now, but I got stuck on somthing and before I could solve it life has interfered. It will be soon though just not sure when.
The Cloud-system resource will not fully support dynamic color shifting or any of that. That should all be a part of Enviro-Torque when it comes out later in the year.
Thanks all for your intrest and support - it keeps me motivated! :-)
06/21/2006 (6:14 am)
Ron and AllI completely agree on previews and delays so I apologize for this one. I wanted to be done by now, but I got stuck on somthing and before I could solve it life has interfered. It will be soon though just not sure when.
The Cloud-system resource will not fully support dynamic color shifting or any of that. That should all be a part of Enviro-Torque when it comes out later in the year.
Thanks all for your intrest and support - it keeps me motivated! :-)
#19
This resource and the constructor tool are my 2 most antisipated resources, Great work man.
06/24/2006 (4:28 pm)
Dont rush. We all understand how helping others can acctually be a pain at times. Do what you need to do and dont stress yourself out. I can always try to figure out all the other implementations till its release. This resource and the constructor tool are my 2 most antisipated resources, Great work man.
#20
07/12/2006 (7:17 am)
Just wanted to post and say thanks for the encouraging progress update! :) From what I've seen, you do excellent work and I second what Ron said -- this is one of my most anticipated resources. I know you'll move forward at your own pace, please let us know if there's anything you'd like us to help out on, and thanks again for working on such an incredible asset for the community. :) 
Torque Owner Funky Diver