Working on Terrain Manager class...
by Bryan Turner · in Torque Game Engine · 04/04/2002 (9:56 am) · 239 replies
SORRY FOLKS!
Looks like the GORPE updates have fallen off the shelf. I do not have a copy of the most recent TM project, but you are welcome to download my old versions here (pre-Torque 1.2):
www.fractalsacpe.org/TorqueTerrain
www.fractalsacpe.org/TorqueTerrainManager.zip
------------------------------------------------------
Hello again.
I got to working on a Terrain Manager class after digging through the terrain rendering code, and it's coming along nicely.
Basically I took Mark F's suggestions and created a TerrainManager class which is the single point of interface for all terrain functions. Moved the relevent structures to it (Material, BlockSize/Mask/Shift, etc). And stubbed the rest of the functions to point to a TerrainBlock. It hit a lot of files to compile/link, but it's working :)
Now I'm getting to the part where I don't understand..
- Should the TerrainBlock remain a SceneObject? I don't think so, since the TerrainManager should be the only thing in the scene.. But taking this step makes the terrain engine *very* unhappy. I'd like some sort of reassurance that this is the right direction.
- How do I specify a vector of data in the mission file datablocks for things like textures, height map files, etc? Basically I want a datablock for TerrainManager which may contain many TerrainBlock datablocks (which contain resource lists for terrain files, textures, etc..).
Some general terrain engine questions:
- What is a 'square' in the terrain engine, and why does *everyone* need it?
- What is a 'grid block' in the terrain engine? How does it differ from a 'square'?
- What is an MPM (material properties map), and why is it passed around with no encapsulation?
Thanks!
--Bryan
Looks like the GORPE updates have fallen off the shelf. I do not have a copy of the most recent TM project, but you are welcome to download my old versions here (pre-Torque 1.2):
www.fractalsacpe.org/TorqueTerrain
www.fractalsacpe.org/TorqueTerrainManager.zip
------------------------------------------------------
Hello again.
I got to working on a Terrain Manager class after digging through the terrain rendering code, and it's coming along nicely.
Basically I took Mark F's suggestions and created a TerrainManager class which is the single point of interface for all terrain functions. Moved the relevent structures to it (Material, BlockSize/Mask/Shift, etc). And stubbed the rest of the functions to point to a TerrainBlock. It hit a lot of files to compile/link, but it's working :)
Now I'm getting to the part where I don't understand..
- Should the TerrainBlock remain a SceneObject? I don't think so, since the TerrainManager should be the only thing in the scene.. But taking this step makes the terrain engine *very* unhappy. I'd like some sort of reassurance that this is the right direction.
- How do I specify a vector of data in the mission file datablocks for things like textures, height map files, etc? Basically I want a datablock for TerrainManager which may contain many TerrainBlock datablocks (which contain resource lists for terrain files, textures, etc..).
Some general terrain engine questions:
- What is a 'square' in the terrain engine, and why does *everyone* need it?
- What is a 'grid block' in the terrain engine? How does it differ from a 'square'?
- What is an MPM (material properties map), and why is it passed around with no encapsulation?
Thanks!
--Bryan
#142
02/15/2003 (5:54 pm)
Currently I'm working on a prototype server framework under Linux. I'm also trying to to get my Torque tree to build under Linux, once that's done I'll be doing another merge of the HEAD source into my source tree. Probably at least a week or two out before I get a chance to take a look at it.
#144
02/16/2003 (7:11 pm)
Not at all! The more folks working with it and refining it the sooner it'll get finished.
#145
nevermind I can't get my head back to a happy place. I tried a fresh jan 1 HEAD + terrain manager and it exploded during runtime at editor launch. If you tell me a good date to pull the head from I can make a patch for current head.
edit:
realized this is based on feb 2002 version of head.. starting again
edit:
that wasn't it either =/
brad
02/16/2003 (7:52 pm)
edit:nevermind I can't get my head back to a happy place. I tried a fresh jan 1 HEAD + terrain manager and it exploded during runtime at editor launch. If you tell me a good date to pull the head from I can make a patch for current head.
edit:
realized this is based on feb 2002 version of head.. starting again
edit:
that wasn't it either =/
brad
#146
The primary issues I've run into is making sure that items such as the player and game don't have a hardcoded dependency on the implementation of TerrainBlock (e.g. when the player samples the material map, or where fluid looks at the heightmap directly).
The way I've thought about implementing this is having a TerrainEngine : SceneObject, and then TerrainBlock : TerrainEngine. TerrainEngine would be the exported interface that everything uses. For the time being, that would allow everything to continue on as it has, but puts in the hooks for generalization to new terrain engines.
03/01/2003 (1:17 pm)
The TerrainManager seems related to some changes I'd like to do, specifically to support plugging in new terrain engines. Have you guys considered that?The primary issues I've run into is making sure that items such as the player and game don't have a hardcoded dependency on the implementation of TerrainBlock (e.g. when the player samples the material map, or where fluid looks at the heightmap directly).
The way I've thought about implementing this is having a TerrainEngine : SceneObject, and then TerrainBlock : TerrainEngine. TerrainEngine would be the exported interface that everything uses. For the time being, that would allow everything to continue on as it has, but puts in the hooks for generalization to new terrain engines.
#147
I've got a couple source files left, then the script files and testing. But... I'm heading out for the evening so it'll be tomorrow at the earliest before I touch it again.
03/01/2003 (1:53 pm)
I am in the process of going through TerrainManager and bringing it up to HEAD as of 2/28/2003. I've been going line by line with Codewrights diff tool and merging in TerrainManager, for the most part it's been straight forward. Once I've got it done I'll produce a patch and put it up on gorpe.com.I've got a couple source files left, then the script files and testing. But... I'm heading out for the evening so it'll be tomorrow at the earliest before I touch it again.
#148
My biggest concern (and I'm sure that shared by everyone else) is forking off the code base significantly only to find out that we're dangling in the wind for future updates because it diverges so much.
03/01/2003 (1:59 pm)
Any idea when/if this is going to make it into HEAD/1.2.0?My biggest concern (and I'm sure that shared by everyone else) is forking off the code base significantly only to find out that we're dangling in the wind for future updates because it diverges so much.
#149
Someone mentioned linux build probs because of compiler complaints, I can help with that too.
I agree with Brian, I think that the waterblock needs to be an aggregate member of the terrain block objs, instead of existing on the same level with tendrils back and forth. I'd be happy to work on this as well, I just didn't want to start hacking away only to have to merge later, and I couldn't find a happy starting point.
As far as I can see back the blocksize has been hardcoded to be 8 (1024 offset) as long as the code has existed, this is reinforced by old posts on the tribes 2 forums.
-brad
03/01/2003 (2:19 pm)
JDS I'd be happy to do the grunt work if I only had a date to use the 'copy-over' files against. Someone mentioned linux build probs because of compiler complaints, I can help with that too.
I agree with Brian, I think that the waterblock needs to be an aggregate member of the terrain block objs, instead of existing on the same level with tendrils back and forth. I'd be happy to work on this as well, I just didn't want to start hacking away only to have to merge later, and I couldn't find a happy starting point.
As far as I can see back the blocksize has been hardcoded to be 8 (1024 offset) as long as the code has existed, this is reinforced by old posts on the tribes 2 forums.
-brad
#150
As for when it'll get into GG's tree? The GG guys can correct me on this but I wouldn't expect this to make it into HEAD without a LOT more work on making it compatible with older missions and it being feature complete.
There are some features, like new mission generation that don't work and must be done by hand. I currently have no plans to make it work because my project will be doing things a little differently and will have different tools for the creation of missions.
If someone would like to take up the cause of making TerrainManager HEADworthy I'll happily turn over the reins.
03/01/2003 (2:36 pm)
Here's the problem, I didn't do the original merge to HEAD on TerrainManager so I can't give you an exact date. Lony did that work, I can tell you his first email to me was on Nov 28th 2002 and he sent me an updated version on 12/2/2002.As for when it'll get into GG's tree? The GG guys can correct me on this but I wouldn't expect this to make it into HEAD without a LOT more work on making it compatible with older missions and it being feature complete.
There are some features, like new mission generation that don't work and must be done by hand. I currently have no plans to make it work because my project will be doing things a little differently and will have different tools for the creation of missions.
If someone would like to take up the cause of making TerrainManager HEADworthy I'll happily turn over the reins.
#151
- introduce an interface class TerrainEngine : SceneObject
- change TerrainBlock to inhereit from TerrainEngine
- remove dependencies on TerrainBlock from all non-terrain files
- remove hardcoded dependencies on TerrainBlock from waterblock/fluid
In addition, I plan on changing the TerrainBlock class so that (hopefully) once and for all you can change the resolution and/or size of the terrain without breaking anything else.
At least, that's my intent. I've posted about this in other threads, and no comments yet, so I'm a bit worried that what I'm doing isn't particularly interesting/relevant or, even worse, has already been done and is pending integration. It's a lot of work that I'd rather not repeat =)
03/01/2003 (2:40 pm)
Ack. Okay, that seems reasonable. What I may end up doing is something incrementally smaller and submitting it for inclusion to HEAD. My general strategy is:- introduce an interface class TerrainEngine : SceneObject
- change TerrainBlock to inhereit from TerrainEngine
- remove dependencies on TerrainBlock from all non-terrain files
- remove hardcoded dependencies on TerrainBlock from waterblock/fluid
In addition, I plan on changing the TerrainBlock class so that (hopefully) once and for all you can change the resolution and/or size of the terrain without breaking anything else.
At least, that's my intent. I've posted about this in other threads, and no comments yet, so I'm a bit worried that what I'm doing isn't particularly interesting/relevant or, even worse, has already been done and is pending integration. It's a lot of work that I'd rather not repeat =)
#152
-brad
03/01/2003 (2:54 pm)
I'm very interested, I was going to kick this thread back up when JDS had time to bring it up to date. Brian if you don't need/want help at the very least I'd be willing to test/confirm.-brad
#153
My biggest concern are the editors. I haven't looked too closely at them, but for my proposed changes, they'll basically only work with the older stuff. What I'm guessing is that I'll have to add a new resource file for each new terrain type which will allow for the appropriate creation of a TerrainEngine subclass.
03/01/2003 (2:58 pm)
Thanks for the offer Brad. The stuff I'm doing isn't exactly Doom 3 rendering, it's just mostly some refactoring. I'll see what I can't get done in a couple days and then maybe let others look at what I've done. The important thing is making sure the initial direction and goals are sensible.My biggest concern are the editors. I haven't looked too closely at them, but for my proposed changes, they'll basically only work with the older stuff. What I'm guessing is that I'll have to add a new resource file for each new terrain type which will allow for the appropriate creation of a TerrainEngine subclass.
#154
My main desire is blocksize and terrain manager, any changes to the renderer you'd probably want to keep to yourself understandably; I'm not looking for free code. Just avoiding duplicated effort. Besides myself and you there are at least 4 other people that are looking for this functionality. The current implementation of terrain manager is pretty functional editorwise according to screenshots and posts. I would have been on this a week ago if I'd been able to get it together, but I was unable to integrate it into any head tree, and instead of spending time on it I lazily decided to wait for the author/and or maintainer. heh.
-brad
03/01/2003 (3:11 pm)
as far as I understand, the editors are 'fresh' i.e. created after the codebase had been stripped of T2 stuff so hopefully it'd be easy to get help/braindumps. My main desire is blocksize and terrain manager, any changes to the renderer you'd probably want to keep to yourself understandably; I'm not looking for free code. Just avoiding duplicated effort. Besides myself and you there are at least 4 other people that are looking for this functionality. The current implementation of terrain manager is pretty functional editorwise according to screenshots and posts. I would have been on this a week ago if I'd been able to get it together, but I was unable to integrate it into any head tree, and instead of spending time on it I lazily decided to wait for the author/and or maintainer. heh.
-brad
#155
Ideally it would be a hierarchy like this:
SceneObject
|
+ TerrainEngine
|
+ TerrainManager
|
+ TerrainBlock
|
+ NewTerrain (or whatever)
Although conceivably TerrainManager would generalize into a TerrainBlock for a 1x1, so TerrainBlock would just be a member of TerrainManager.
03/01/2003 (3:22 pm)
I won't be integrating the Terrain Manager system, since I think it's trying to accomplish something different what I'm doing. I _think_ that it's basically a way to have the existing terrain engine extended to an MxN grid of terrains. Which is cool, but what I'm trying to do is replace the terrain engine _entirely_.Ideally it would be a hierarchy like this:
SceneObject
|
+ TerrainEngine
|
+ TerrainManager
|
+ TerrainBlock
|
+ NewTerrain (or whatever)
Although conceivably TerrainManager would generalize into a TerrainBlock for a 1x1, so TerrainBlock would just be a member of TerrainManager.
#156
is the chunked LOD the same as that fellow did and used in the snowboarding game?
-brad
03/01/2003 (3:37 pm)
That's the idea, from what I understand about the terrain manager that if unused it just assumes 1x1 (or thats the idea, not yet put in yet =D). What you are doing makes sense to me now, I read your previous posts too fast.is the chunked LOD the same as that fellow did and used in the snowboarding game?
-brad
#157
The two primary limitations that it suffers from are:
1. Collision detection is no longer a trivial heightmap query with interpolation. You have to descend into the quad tree and do a more rigorous search against the RTIN triangles. It sounds worse than it really is, probably a page of code.
2. You cannot dynamically alter the terrain, e.g. with craters, etc. That's a trade off I'm more than willing to live with, especially given that most other terrain rendering algorithms don't support this either.
03/01/2003 (3:43 pm)
Yes, chunked LOD is the technique described by Thatcher Ulrich. It's probably one of the simplest and most effective terrain rendering techniques today. It has very good performance on high performance video cards (since you can cache data in vertex buffers), all the computations are done during a massive precomputation step, it allows for background loading of resources, it has no significant artifacts, etc.The two primary limitations that it suffers from are:
1. Collision detection is no longer a trivial heightmap query with interpolation. You have to descend into the quad tree and do a more rigorous search against the RTIN triangles. It sounds worse than it really is, probably a page of code.
2. You cannot dynamically alter the terrain, e.g. with craters, etc. That's a trade off I'm more than willing to live with, especially given that most other terrain rendering algorithms don't support this either.
#159
After merging in TerrainManager from the rar file with the latest HEAD I ended up with a non-working copy of Torque. Yesterday I reverted back and started adding TerrainManager code again this time using my working copy as a reference. I'm going slowly and making sure each change is correct as I go so it's taking a bit longer. Right now I've got a handful of files to go and some testing to do.
This will probably be the last version of TerrainManager that even attempts backwards compatibility with the Tribes2 files.
03/03/2003 (6:53 am)
Just an fyi:After merging in TerrainManager from the rar file with the latest HEAD I ended up with a non-working copy of Torque. Yesterday I reverted back and started adding TerrainManager code again this time using my working copy as a reference. I'm going slowly and making sure each change is correct as I go so it's taking a bit longer. Right now I've got a handful of files to go and some testing to do.
This will probably be the last version of TerrainManager that even attempts backwards compatibility with the Tribes2 files.
#160
Passing by to check on the TerrainManager project. It looks like things are in a bit of chaos at the moment.
J. Donavan Stanley:
I've read repeated attempts by people to merge into HEAD without success. Has the engine changed that much under the hood? My last update was a few weeks before 1.1.2 (?), it's unfortunate that I left it hanging on HEAD at the time instead of a known build. If there are any files that you're having difficulty merging, please let me know. I would like to help put this project back on track.
Brian Hook:
The goals of TerrainManager and yourself are in alignment. You will be replicating much of the TerrainManager's work in order to make your extensions. This is mainly due to the un-encapsulated functions and structures used by the rest of the system. TerrainManager acts as a thin layer maintaining an abstraction for these operations.
For instance, the engine can call a Height(x,y) function to get the height of a location from outside the TerrainBlock, and it will be multiplexed to the correct object (ie: your object instead of the normal TerrainBlock). From which you can perform the correct operation and return the height. All external functions (ie: collision & lighting) use this function to obtain information about the terrain.
The second layer of TerrainManager was to be a plug-in rendering engine, but I never got around to that part. It sounds like this is the piece you are most interested in. I think you will find everything you need from the engine is already available to you in TerrainManager, and it also handles all the interfaces you don't care about. I would encourage you to extend TerrainManager instead of reinventing half of the wheel.
The NxM grid feature was the simplest extension (without replacing the terrain algorithm) to get a multi-landscape Torque. Once this level is reached, the interfaces were in place to have the entire back end system overhauled with a new engine (which is what you are doing).
Please take the time to look at TerrainManager before you decide to start from scratch, I think you will be happy with the interfaces.
--Bryan
bryan.turner@pobox.com
03/03/2003 (10:29 am)
Hello,Passing by to check on the TerrainManager project. It looks like things are in a bit of chaos at the moment.
J. Donavan Stanley:
I've read repeated attempts by people to merge into HEAD without success. Has the engine changed that much under the hood? My last update was a few weeks before 1.1.2 (?), it's unfortunate that I left it hanging on HEAD at the time instead of a known build. If there are any files that you're having difficulty merging, please let me know. I would like to help put this project back on track.
Brian Hook:
The goals of TerrainManager and yourself are in alignment. You will be replicating much of the TerrainManager's work in order to make your extensions. This is mainly due to the un-encapsulated functions and structures used by the rest of the system. TerrainManager acts as a thin layer maintaining an abstraction for these operations.
For instance, the engine can call a Height(x,y) function to get the height of a location from outside the TerrainBlock, and it will be multiplexed to the correct object (ie: your object instead of the normal TerrainBlock). From which you can perform the correct operation and return the height. All external functions (ie: collision & lighting) use this function to obtain information about the terrain.
The second layer of TerrainManager was to be a plug-in rendering engine, but I never got around to that part. It sounds like this is the piece you are most interested in. I think you will find everything you need from the engine is already available to you in TerrainManager, and it also handles all the interfaces you don't care about. I would encourage you to extend TerrainManager instead of reinventing half of the wheel.
The NxM grid feature was the simplest extension (without replacing the terrain algorithm) to get a multi-landscape Torque. Once this level is reached, the interfaces were in place to have the entire back end system overhauled with a new engine (which is what you are doing).
Please take the time to look at TerrainManager before you decide to start from scratch, I think you will be happy with the interfaces.
--Bryan
bryan.turner@pobox.com
Torque Owner Brett Fattori
Is there a chance a new version will be available soon that is compatible with the latest HEAD?
Thanks!
- Brett