big terrains
by Matthew Shapiro · in Torque Game Engine · 04/08/2001 (8:23 pm) · 40 replies
I understand by reading through these posts that the game uses large "tiles" that go on adn on and on etc... but with the terrain editor do you edit the main tile or do you edit the WHOLE map? I ask this because I don't mind the terrain bein the same but for an MMORPG i would love to just have one big map and be able to make some places turn into a desert like place, some a forest, some mountanous, etc... I'm tired heh so this might not have been able to be understood so if not i'll clarify in the morning after i get some sleep :)
#22
There is one simple reason I'm not very keen on the "one big map" proposal.
Map load time.
Tribes 2 maps already take painfully long to load, and any RPG-type game will have far more terrain objects than T2. The load times at startup might be insane.
04/11/2001 (9:06 am)
Matthew S:There is one simple reason I'm not very keen on the "one big map" proposal.
Map load time.
Tribes 2 maps already take painfully long to load, and any RPG-type game will have far more terrain objects than T2. The load times at startup might be insane.
#23
04/11/2001 (12:06 pm)
Wes, this maybe but you only have to load it once, so if you go in a cave, dungeon, flying above a town, go into a shop etc... you don't load anything! 
#24
04/11/2001 (12:45 pm)
And every user has 384 mb ram too. 
#25
04/11/2001 (12:51 pm)
You could just have the server maintain the huge map or mulitple servers maintaining multiple maps and then simply (simply- hah!) stream the parts of the map and objects that the user needs to them. The user would have a reasonable load time for the initial area (same time it would take to load a small map on their computer) around them but if the rest is sent in small enough increments then they would never have to experience a load time again unless they hop to a radically different part of the world.
#26
Also I would love to do the idea about streaming maps because this would allow us to update the map without having to put out patches, which means patches are only for models,sounds,or menus which will most likey shorten the patches and make the smaller BUT i have no idea how i would stream the map, considering i don't know the map algorithm nor do i know how to make it so that the client recieves and puts the information in the right spot!
04/11/2001 (6:11 pm)
you don't need 384 megs of ram! the tribes game (at least 1) does the same thing and it tells you when your going out too far but basically it goes on for infinity using the tile thing!Also I would love to do the idea about streaming maps because this would allow us to update the map without having to put out patches, which means patches are only for models,sounds,or menus which will most likey shorten the patches and make the smaller BUT i have no idea how i would stream the map, considering i don't know the map algorithm nor do i know how to make it so that the client recieves and puts the information in the right spot!
#27
04/12/2001 (6:35 am)
It would take major, major engine work to impliment this streaming tile thing. It's not an efficent or easy thing to do to add more data dynamically to a map. I would like to see it done, however I still maintain that this is not a probable way to do this.
#28
http://www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=482
--Bryan
04/12/2001 (9:38 am)
This post seems to be trailing off-topic, so I started a new thread to discuss how to actually implement large terrains using paging/streaming.http://www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=482
--Bryan
#29
If two clients, ZeroCool and Tommy, are near the same point in space according to the server, they are both processing the same seed and set of equations; thus, they see the exact same terrain. While Zero and Tom stay within "view," so to speak, of the immediate terrain patch, the terrain generator does not need to be called to regenerate old heightmaps. But if Zero and Tom start heading north together, approaching the boundry of the current terrain patch, another new terrain needs to be generated. The signifigance of generating new (different) terrain is that it can be done with a new arbitrary seed and set of equations. So Zero's and Tom's starting terrain could be rocky and mountainous, but if they head north, the mountains will blend into rolling hills (dictated by the new seed and heightmap).
Create another algorithm that associates texturing rules (slope, height, x/y globe position even) for each algorithmically generated terrain patch and provide a set of prefabricated "interiors" to be arbitrarily placed throughout the realm, and you've got a pretty decent world to play with. Algorithmically (with a arbitrary seed, mind you) place vegetation, rocks, and other objects and you're planet will start looking pretty sharp. The important thing is that you separate your normal rendering process from the terrain generator process. If you'd like to provide seemless transitions between terrain patches, you can't have your players halt at the patch boundry, generate the next terrain, and move on. The client should detect that you're approaching the boundry of a terrain patch, and start generating the next terrain patch as a background process before the new terrain patch comes into view. Hell, you could take a step further and recalculate a moving sun's effect on terrain shadows as a background process that spits out a new terrain lightmap every 5 minutes er so.
Combine the whole thing with a Dynamic Load Balancing Server Model and you too can kick more ass.
So how's that for large terrains?
--Timothy "Drake" Ford
04/18/2001 (1:12 am)
Is anyone familiar with algorithimically generated terrain? If you aren't, and you have a copy of Tribes 2, jump into the editor (editor.open();) and go to the terraformer. Most of those heightmap generators use algorithmically generated terrain. Disreguard terrain texturing for the time being. The output of a hightmap generator is based on the equation done to it (fractal, smooth, blend, erosion, brownian noise, etc.) and the seed fed to the algorithm. Consequentially, one could set up a system where the terrain rendered to your screen at a certain point in a "Large Terrain" world is based on an arbitrary seed and set of equations in place of a set heightmap. Most of you can agree that a seed value and set of equations has a much smaller memory footprint than your standard hightmap. Furthermore, the memory size of a heightmap patch increases linearly with the size of the patch while the size of the seed + equations (limited to a certain patch size) doesn't really increase in memory size at all. The importance of small memory footprint is that your terrain generator algorithm will reside snugly within level 1 and 2 CPU caches. Trying to fish a massive heightmap out of primary memory can be one order of magnitude slower than accessing cache.If two clients, ZeroCool and Tommy, are near the same point in space according to the server, they are both processing the same seed and set of equations; thus, they see the exact same terrain. While Zero and Tom stay within "view," so to speak, of the immediate terrain patch, the terrain generator does not need to be called to regenerate old heightmaps. But if Zero and Tom start heading north together, approaching the boundry of the current terrain patch, another new terrain needs to be generated. The signifigance of generating new (different) terrain is that it can be done with a new arbitrary seed and set of equations. So Zero's and Tom's starting terrain could be rocky and mountainous, but if they head north, the mountains will blend into rolling hills (dictated by the new seed and heightmap).
Create another algorithm that associates texturing rules (slope, height, x/y globe position even) for each algorithmically generated terrain patch and provide a set of prefabricated "interiors" to be arbitrarily placed throughout the realm, and you've got a pretty decent world to play with. Algorithmically (with a arbitrary seed, mind you) place vegetation, rocks, and other objects and you're planet will start looking pretty sharp. The important thing is that you separate your normal rendering process from the terrain generator process. If you'd like to provide seemless transitions between terrain patches, you can't have your players halt at the patch boundry, generate the next terrain, and move on. The client should detect that you're approaching the boundry of a terrain patch, and start generating the next terrain patch as a background process before the new terrain patch comes into view. Hell, you could take a step further and recalculate a moving sun's effect on terrain shadows as a background process that spits out a new terrain lightmap every 5 minutes er so.
Combine the whole thing with a Dynamic Load Balancing Server Model and you too can kick more ass.
So how's that for large terrains?
--Timothy "Drake" Ford
#30
04/18/2001 (2:45 am)
you mean to keep the terrain basically randomly generated?
#31
I did much of the work on that terrain generator and yes 95% of the T2 mission areas are algorithmically generated and textured. The terrain generation algorithms I used in the V12 tile on a 256x256 square but all can be easily altered to generate an infinite ever changing landscape. I think there are always going to be areas that you will want to customize by hand for your missions, towns, race tracks, whatever. So while I have had the same thoughts of 100% algorithmically generated terrains, in practice it just doesn't work. So we need to think about how to compress the stuff.
The terrain data is not that large right now and with a little work could be highly compressed with wavelets and streamed to the clients. What you need to be careful of is errors introduced by the compression algorithm. A half meter error in the wrong place could leave a corner of a building floating in the air. Not a pretty sight.
I have been wanting to put some time into researching algorithmically placed and created objects like trees, shrubs, rocks, etc. You'll notice I did add several L-system articles in the GarageGames resource database a while back. I think algorithmically placing and generating these meshes holds a lot of promise. One of these days Tim and I will get a few free cycles to do some experimentation... along with the other 100 ideas we have yet to explore.
--Rick
04/18/2001 (6:01 am)
More algorithmic than random.I did much of the work on that terrain generator and yes 95% of the T2 mission areas are algorithmically generated and textured. The terrain generation algorithms I used in the V12 tile on a 256x256 square but all can be easily altered to generate an infinite ever changing landscape. I think there are always going to be areas that you will want to customize by hand for your missions, towns, race tracks, whatever. So while I have had the same thoughts of 100% algorithmically generated terrains, in practice it just doesn't work. So we need to think about how to compress the stuff.
The terrain data is not that large right now and with a little work could be highly compressed with wavelets and streamed to the clients. What you need to be careful of is errors introduced by the compression algorithm. A half meter error in the wrong place could leave a corner of a building floating in the air. Not a pretty sight.
I have been wanting to put some time into researching algorithmically placed and created objects like trees, shrubs, rocks, etc. You'll notice I did add several L-system articles in the GarageGames resource database a while back. I think algorithmically placing and generating these meshes holds a lot of promise. One of these days Tim and I will get a few free cycles to do some experimentation... along with the other 100 ideas we have yet to explore.
--Rick
#32
04/18/2001 (6:31 am)
When I was still considering doing a MMORPG with the V12, I considered what Pat had said earliar. Due to my limited resources, I knew I couldn't host huge server farms. So I was going to distribute the server for free, maybe allow a little in game advertising (allowing servers to turn a buck), and create what I have dubbed as Virtual Server Farms. Each server would locate themselves on a map, and a central server would group them and create server farms that weren't actually server farms. Using this I could split V12 maps among the servers, making the world infinately expandable. The way AC works is a lot different from the way the V12 is currently set up. I suggest an EQ aproach, as it is simpler, and allows for the world to be changed merely by adding a new server and map.
#33
I wrote a real-time procedural terrain generator ala Timothy's description using a Poisson-distribution random point sampling algorithm. I chose this method because it was extremely low CPU hit, and you could sample it for a specific X,Y if desired (instead of some methods which require building entire height maps before you could sample an X,Y), you could also use Perlin Noise, but it requires more CPU.
After playing with this, I decided that it wasn't dynamic enough. I really wanted the terrain to be user-modifiable. So I designed a system which takes input from a fractal function and a height map. The height map has a special feature though: The heights represent offsets from the fractal map, rather than implicit heights. This allows a world builder to wipe an area flat to place a building, or add hills, streams, canyons, etc.. as desired. This 'displacement map' was then stored to disk along with its coordinates in world-space.
Also, my desire to abstract the terrain representation away from the actual CLOD & rendering led me to a paging system (described here). Using this system, I could abstract the actual implementation of the height data. Thus it could be paged off a disk, generated on-the-fly, or streamed over a network. More importantly, it could do all three at once!
*takes a breath*
So... With all of that done, I wanted to allow people to run around in it, not just one or two. I wanted 10,000+ people all interacting in the world at once. Since I don't have the time or money to build a server farm, I'm designing a system ala Trevor's post. A fully-scalable hierarchical peer to peer system such that everyone who plays the game becomes a node in the world. Depending on certain factors (line speed, reliability, storage capacity), the users would be promoted to more responsible positions in the hierarchy.
--Bryan
04/18/2001 (9:34 am)
Wow, I love it when a conversation turns directly into my line of research! :)I wrote a real-time procedural terrain generator ala Timothy's description using a Poisson-distribution random point sampling algorithm. I chose this method because it was extremely low CPU hit, and you could sample it for a specific X,Y if desired (instead of some methods which require building entire height maps before you could sample an X,Y), you could also use Perlin Noise, but it requires more CPU.
After playing with this, I decided that it wasn't dynamic enough. I really wanted the terrain to be user-modifiable. So I designed a system which takes input from a fractal function and a height map. The height map has a special feature though: The heights represent offsets from the fractal map, rather than implicit heights. This allows a world builder to wipe an area flat to place a building, or add hills, streams, canyons, etc.. as desired. This 'displacement map' was then stored to disk along with its coordinates in world-space.
Also, my desire to abstract the terrain representation away from the actual CLOD & rendering led me to a paging system (described here). Using this system, I could abstract the actual implementation of the height data. Thus it could be paged off a disk, generated on-the-fly, or streamed over a network. More importantly, it could do all three at once!
*takes a breath*
So... With all of that done, I wanted to allow people to run around in it, not just one or two. I wanted 10,000+ people all interacting in the world at once. Since I don't have the time or money to build a server farm, I'm designing a system ala Trevor's post. A fully-scalable hierarchical peer to peer system such that everyone who plays the game becomes a node in the world. Depending on certain factors (line speed, reliability, storage capacity), the users would be promoted to more responsible positions in the hierarchy.
--Bryan
#34
04/18/2001 (10:33 am)
Using your research, how hard would it be to upgrade the V12 technology to it? Id imagine you would have to re-write the entire map code as well as a portion of the network code. Or in your mind would it be a simple fix? I wonder if you could blend the two together to create offline areas where the player could do certain things, or maybe a safe haven, or the void, etc. I always wondered why certain things in some MMORPG's take place online when they don't have to, even if it is for verification to prevent cheating.
#35
Right.. so now that that's explained, I imagine the render module for the Tribes 2 engine can be further dissected into its bsp module, lighting module, terrain module and anything else they packed in there. Those modules may also be shoved into another linked list that is iterated upon when the render module's update() function is called.
Assuming all these components are broken down into "black-box" modules (ie. give it an input, and it returns your output and you don't care what goes on inside), one could easily isolate the terrain module, and make the conversions there without affecting any other part of the program (in theory).
-- Timothy "Drake" Ford
"I'd love to live in theory, everything works in theory."
04/18/2001 (11:21 am)
We'd really have to look at how the code is structured. Most game engines these days are designed like an operating system scheduler. Your average game has one module for each "department" so to speak. So you got your sound, physics, network, texture/mesh/level database, game mechanics, and render modules and so on to work with. Each module is a C++ object inherited from one main C++ object. The Parent object only passes on one real function to all its children, the "update()" member function. The parent object acts like a mini-os and stores each module into a massive linked list (or some other neat little data structure) in the order each module must be executed. The mini-os then goes down the list, running each modules' update() function (assuming the update() function does all the work required for that module to render the current frame or update within the server's network window). The event loop of the game engine basically runs through that linked list over and over and over again. The great thing about this model is that you can add and remove modules from (ie. schedule them into and out of) the mini-os linked list. So if you don't need any network updates to render this frame, take that module out of the list for this iteration.Right.. so now that that's explained, I imagine the render module for the Tribes 2 engine can be further dissected into its bsp module, lighting module, terrain module and anything else they packed in there. Those modules may also be shoved into another linked list that is iterated upon when the render module's update() function is called.
Assuming all these components are broken down into "black-box" modules (ie. give it an input, and it returns your output and you don't care what goes on inside), one could easily isolate the terrain module, and make the conversions there without affecting any other part of the program (in theory).
-- Timothy "Drake" Ford
"I'd love to live in theory, everything works in theory."
#36
The network code from my research project is only in its infancy. I have a Tribes2-like implementation on my projects site (www.fractalscape.org), but there's much more to be done.
As for the Terrain/Mapping work, it depends on how V12 is built (ala Timothy's description). My research engine could be polished up and added as a module to the V12 code, but the difficulty of that conversion process depends on the internals of V12.
It would be really nice to add some of these features onto the V12 codebase for use in the myriad of MMOGs people are planning. I'm also interested in the GG designs for hosting online games. Perhaps a single network architechture could be designed that would fufill the needs of action players, MMOGs, and turn-based systems? This would certainly simplify the backend server systems GG would have to handle. Also, players would only have to logon once to play any of the games on the GG network. (I'm sure this gets complicated when new protocol versions start arriving.. oh well).
--Bryan
04/18/2001 (12:19 pm)
Trevor,The network code from my research project is only in its infancy. I have a Tribes2-like implementation on my projects site (www.fractalscape.org), but there's much more to be done.
As for the Terrain/Mapping work, it depends on how V12 is built (ala Timothy's description). My research engine could be polished up and added as a module to the V12 code, but the difficulty of that conversion process depends on the internals of V12.
It would be really nice to add some of these features onto the V12 codebase for use in the myriad of MMOGs people are planning. I'm also interested in the GG designs for hosting online games. Perhaps a single network architechture could be designed that would fufill the needs of action players, MMOGs, and turn-based systems? This would certainly simplify the backend server systems GG would have to handle. Also, players would only have to logon once to play any of the games on the GG network. (I'm sure this gets complicated when new protocol versions start arriving.. oh well).
--Bryan
#37
Ive only just started reading over the threads while awaiting the V12 release.
A group of friends and I have been working on a Multiplayer (not really mmpog) 3drpg. At the moment it is aimed at lan play. We have done some tests with other direct3d based engines.
We have tossed around different suggestions for doing the terrain. We want to give V12 a go after seeing the features it supports. As it stands, we will most likely do the worldmap EQ style with zones.
I have noticed that T2 actually takes a little bit of time loading a level. So we decided to make 1 level per zone and have them swap between each zone.
I do not think that making the landmass 1 map is really going to work (not without some major code additions).
I would love to see an alternative that allows players to walk seamlessly from one side to the other.
If anyone does eventually develop a method to do this, I would love to get together to share code etc.
Caliban
04/22/2001 (10:39 pm)
Hi guys,Ive only just started reading over the threads while awaiting the V12 release.
A group of friends and I have been working on a Multiplayer (not really mmpog) 3drpg. At the moment it is aimed at lan play. We have done some tests with other direct3d based engines.
We have tossed around different suggestions for doing the terrain. We want to give V12 a go after seeing the features it supports. As it stands, we will most likely do the worldmap EQ style with zones.
I have noticed that T2 actually takes a little bit of time loading a level. So we decided to make 1 level per zone and have them swap between each zone.
I do not think that making the landmass 1 map is really going to work (not without some major code additions).
I would love to see an alternative that allows players to walk seamlessly from one side to the other.
If anyone does eventually develop a method to do this, I would love to get together to share code etc.
Caliban
#38
www.garagegames.com/index.php?sec=mg&mod=resource&page=result&qtm=Terrain%20LOD
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=258
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=321
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=3319
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=3414
(shameless self-promotion)
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=482
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=445
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=1821
(no GG stuff, but usefull)
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=4029
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=3312
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=2471
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=2341
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=1670
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=1100
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=2293
--Bryan
03/31/2002 (8:00 pm)
Stitching together a 'fabric' of terrain posts... These are posts which contain significant responses from the GG crew:www.garagegames.com/index.php?sec=mg&mod=resource&page=result&qtm=Terrain%20LOD
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=258
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=321
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=3319
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=3414
(shameless self-promotion)
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=482
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=445
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=1821
(no GG stuff, but usefull)
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=4029
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=3312
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=2471
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=2341
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=1670
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=1100
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=2293
--Bryan
#39
Did anyone actualy get a non-zoned approach up and running?
Anything in the past 2 years?
02/07/2005 (5:01 pm)
Did this topic die?Did anyone actualy get a non-zoned approach up and running?
Anything in the past 2 years?
#40
Depending on how much time you want to sink into the development. The fastest.. easiest approach is a zone based approach just like everquest.
02/07/2005 (5:44 pm)
Look into the other posts and you will find your answer.Depending on how much time you want to sink into the development. The fastest.. easiest approach is a zone based approach just like everquest.
Torque Owner Matthew Shapiro