big terrains
by Matthew Shapiro · in Torque Game Engine · 04/08/2001 (8:23 pm) · 40 replies
I understand by reading through these posts that the game uses large "tiles" that go on adn on and on etc... but with the terrain editor do you edit the main tile or do you edit the WHOLE map? I ask this because I don't mind the terrain bein the same but for an MMORPG i would love to just have one big map and be able to make some places turn into a desert like place, some a forest, some mountanous, etc... I'm tired heh so this might not have been able to be understood so if not i'll clarify in the morning after i get some sleep :)
#2
04/09/2001 (4:22 am)
Woah woah here...head out of the clouds and think a little bit here. You chunk the whole game into one map, how are you doing to deal with an expanding playerbase and world overcrouding, just add on to the map? How are you going to deal with 500 players in that one area? This is NOT a good idea in any respect. It totally kills expandability to multiple servers, kills easy world expandability, will make your life extremely difficult to debug any problems. What you probably want is to make the player THINK it's one big map...IE the Ultima Online model vs EverQuest.
#3
04/09/2001 (5:49 am)
Your exactly right..... i said, its "could" be one big map. But, i totaly agree with you. One Big Map = Havoc. I mean, Tribes 2 already lags my system down when 30 people are fight near by (I run a 1.1Ghz T-Bird, 256mb RAM, and a 64mb GeForce 2 GTS). And if just 30 people would slow it down, imagine 300 - 700 fighting all at once when someone who is running a 500 Mhz, 128mb RAM, and maybe a 16mb Graphic card. It would suck, big time. With the Ever Quest zone thing, the game runs smoother. But, its a huge pain, server side, and client side. If you could do what Asheron's Call does, you'd be cool. I don't play Asheron's Call (my little bro plays it, I'm an EQ fan), but from what i've seen, the world is one big map. I'm not sure if it streams sections of it as you get closer or what(which is possable, in programming, anything is possable). If you could get your MMORPG to run like Asheron's Call, and maybe made each continent a map, you'll be fine. Or you could do it the old fashion EQ way, and make tons of zones, which would then take tons of servers to run your game. Either one.
#4
04/09/2001 (6:10 am)
The solution I'd impliment, do it EQ style...sort of. Each server program runs one zone, you can run multiple zones on one machine however. (Multi-processor would be very handy) When a client gets near a zone border instruct the client to send it's packets to both servers so people in the destination zone can see the client approaching, and then once he passes a certain line on the other side, he no longer sends information to the previous zone. (That is pretty rough a description but I don't have time for a longer one currently)
#5
The landscapes and static objects (trees, rocks, etc) are all persisted on the client. Every month when a new update to the game is propagated, each client downloads a patch to the data before entering the game.
Dynamic objects like doors, monsters, etc are streamed to the client as they move about.
That's all pretty unspectacular. The most impressive piece of the system is the dynamic load balancing. They periodically subdivide the terrain based on population grouping and assign each subdivision to a server.
So imagine a huge square terrain area (top down view of the terrain). Lets say for the purpose of example that most of the population is grouping near the upper right of the square. Also, lets say we have 3 servers dedicated to handling this area.
The subdivision works just like octree techniques... you split the world along an axis, then position the split based on the population of each side. Causing a large area with few people to be handled by 1 server and small, densely populated areas to get servers dedicated to them.
So in the example above, 1 server might handle 75% of the terrain since only 10% of the population resides there, and another server might handle the population-dense upper-right-corner, even though it only represents 10% of the terrain. (The 3rd server would handle the leftovers, the area adjacent to the upper corner)
Obviously, V12 won't be able to do this out of the box. I'll be very interested in any projects which hope to implement this sort of technology in V12, since there is a heap of obvious challenges in getting such an approach to work.
04/09/2001 (8:18 am)
I have some inside knowledge on how engine which powers Asherons Call works, so I thought I'd share.The landscapes and static objects (trees, rocks, etc) are all persisted on the client. Every month when a new update to the game is propagated, each client downloads a patch to the data before entering the game.
Dynamic objects like doors, monsters, etc are streamed to the client as they move about.
That's all pretty unspectacular. The most impressive piece of the system is the dynamic load balancing. They periodically subdivide the terrain based on population grouping and assign each subdivision to a server.
So imagine a huge square terrain area (top down view of the terrain). Lets say for the purpose of example that most of the population is grouping near the upper right of the square. Also, lets say we have 3 servers dedicated to handling this area.
The subdivision works just like octree techniques... you split the world along an axis, then position the split based on the population of each side. Causing a large area with few people to be handled by 1 server and small, densely populated areas to get servers dedicated to them.
So in the example above, 1 server might handle 75% of the terrain since only 10% of the population resides there, and another server might handle the population-dense upper-right-corner, even though it only represents 10% of the terrain. (The 3rd server would handle the leftovers, the area adjacent to the upper corner)
Obviously, V12 won't be able to do this out of the box. I'll be very interested in any projects which hope to implement this sort of technology in V12, since there is a heap of obvious challenges in getting such an approach to work.
#6
04/09/2001 (8:30 am)
Dynamic subdivision...now THAT is an idea I didn't think of before. Damn fine idea, thanks for posting that. Yeah, that'll definatly take some modifications, but I think it'd be worth it.
#7
Also i highly doubt you can't split the job between networked computers!
04/09/2001 (9:34 am)
Actully 1 big map does not equal havoc IF you do it right. bandwidth lag cna easily be handled by only sending information to a player of objects that are VISIBLE and a certain distance close to him. Now i really don't want to do zoning but if you want to get rid of lag due to processer speed buy a 4 processer motherboard, buy lots of ram, put 4 1ghz processors in there and you should be set for a good amount of time. YOu also have to take into consideration that i'm a 16 year old with 16 year old friends who just want to do an MMORPG for the fun of it, not for profit. So at first i won't have many users and the only reason i'll be charging is for server maintanence, but that's once a) our server can't handle it anymore or b) we're out of beta with enough people.Also i highly doubt you can't split the job between networked computers!
#8
I dislike (intensely) the EQ method. They put too much 'stuff' into some zones (eg: Kelethin), and practically nothing into others (eg: the sea). If EQ used a no-zone method with dynamic load banancing, things would be much nicer. 400 users in one zone really killed the experience sometimes. Especially with /shout. I recall EQ uses 4-6 zones per server usually designed to load-ballance the servers (ie: Kelethin + sea + other low-use zone = one server)
UO is not too shabby in terms of world handling, but they're really killed by TCP issues. If they moved the updates to UDP the game would be vastly smoother in my opinion. Also, the world is much too small for the number of users on a shard.
I'd really like to see a project under V12 that allowed dynamic load-balancing among the server farm, and support for streamed terrain to a local cache. I believe this would fix all the problems: When areas get full they would be farmed off to another server.
When areas are added, they simply 'appear' to the users as they get closer. Thereafter, the terrain is in the user's cache and does not have to be resent. This would eliminate some of the need for 'patches', as they would simply stream the new content as the user plays.
As for the original question of this thread: I would not use _one_ large map. Instead use square patches of some size (say 256x256) which have only one set of textures each. This would help both streaming and texture generation for the world. Managing all of this for one map would be a headache. I have a CLOD terrain engine which does this already, including texture LOD and paging/streaming, which works very well.
Ramblin' on...
--Bryan
04/09/2001 (10:00 am)
After playing EQ, UO, and AC (demo'd by a friend). I can say AC's world-handling is much better. I didn't realize they used dynamic subdivisions, but that makes it even cooler.I dislike (intensely) the EQ method. They put too much 'stuff' into some zones (eg: Kelethin), and practically nothing into others (eg: the sea). If EQ used a no-zone method with dynamic load banancing, things would be much nicer. 400 users in one zone really killed the experience sometimes. Especially with /shout. I recall EQ uses 4-6 zones per server usually designed to load-ballance the servers (ie: Kelethin + sea + other low-use zone = one server)
UO is not too shabby in terms of world handling, but they're really killed by TCP issues. If they moved the updates to UDP the game would be vastly smoother in my opinion. Also, the world is much too small for the number of users on a shard.
I'd really like to see a project under V12 that allowed dynamic load-balancing among the server farm, and support for streamed terrain to a local cache. I believe this would fix all the problems: When areas get full they would be farmed off to another server.
When areas are added, they simply 'appear' to the users as they get closer. Thereafter, the terrain is in the user's cache and does not have to be resent. This would eliminate some of the need for 'patches', as they would simply stream the new content as the user plays.
As for the original question of this thread: I would not use _one_ large map. Instead use square patches of some size (say 256x256) which have only one set of textures each. This would help both streaming and texture generation for the world. Managing all of this for one map would be a headache. I have a CLOD terrain engine which does this already, including texture LOD and paging/streaming, which works very well.
Ramblin' on...
--Bryan
#9
04/09/2001 (10:05 am)
The problem with sending them the map as they go along is that you need to know how the maps are coded, and make it lag efficient when streaming the map. I had this idea before but... Also if you do this i see no reason why you can't have one big map!
#10
04/09/2001 (10:31 am)
Well the problem with streaming the map is that you have to do LOD and LOS calculations for every player server side. The multiple server approach is the best, no matter how you decide to impliement "zones" just because while multi-processor machines are helpful, all it allows you to do is run 2 threads at the same time (or 4 if you have a 4 processor machine)...you still have to share the RAM. The best way to impliment it, in any situation, is the multiple server method because you could either have a server farm, or have 3 zones running on the same computer, or whatever. It wouldn't matter to the server. Just feed it an IP of the next "zone" over and it'll be happy.
#11
In effect, the streamed-map IS one huge map. This gets into what it means to 'stream' or 'page' a file. If you store your map as one large BMP, for instance, then each 'line' of the map would be really long. For a 100K x 100K world, each line is 100K long.
To page in a 256x256 section of the map, you would have to do 256 separate fread() operations and combine them together into one tile. (you certainly can't keep a 100K x 100K terrain in memory all at once)
So.. as I said before, 'one big map' gives headaches, but the 'appearance' of one big map can still be given to the user (ala Asheron's Call).
------------
Pat,
I may be off-base here, but why would streaming be a problem with LOS calculations? The server knows what tile the user is on now, and all the surrounding tiles. If the user doesn't have one of those tiles, send it to them. There are plenty of more intelligent methods (mip-mapping terrain for instance) which could reduce the problems of users entering into new territory before the streaming caught up.
--Bryan
04/09/2001 (1:10 pm)
Matthew,In effect, the streamed-map IS one huge map. This gets into what it means to 'stream' or 'page' a file. If you store your map as one large BMP, for instance, then each 'line' of the map would be really long. For a 100K x 100K world, each line is 100K long.
To page in a 256x256 section of the map, you would have to do 256 separate fread() operations and combine them together into one tile. (you certainly can't keep a 100K x 100K terrain in memory all at once)
So.. as I said before, 'one big map' gives headaches, but the 'appearance' of one big map can still be given to the user (ala Asheron's Call).
------------
Pat,
I may be off-base here, but why would streaming be a problem with LOS calculations? The server knows what tile the user is on now, and all the surrounding tiles. If the user doesn't have one of those tiles, send it to them. There are plenty of more intelligent methods (mip-mapping terrain for instance) which could reduce the problems of users entering into new territory before the streaming caught up.
--Bryan
#12
Maybe I'm not understanding what you are proposing.
04/09/2001 (1:36 pm)
It would depend on the application I suppose. For each player if you used V12, you'd need to know where they were, and where they were looking at each instant to make sure they had all the map data they needed, and you could cull what you didn't need. If there was a zoom option they'd need even more. Maybe I'm not understanding what you are proposing.
#13
Not to crush anyones dreams here, but MMORPGS are too large to just for fun. I twould be nice that a small "Garage Games" company could do one, but its highly unlikely.
Possibily mentioned before; All MMOGs use zones. Call them what you want and try to cover them up, they are still zones.
The Turbine 2.0 engine(Asherons Call2) handels the world much like Asherons Call did. However, instead of covering up zones with portals, the "portal"(which was coincidently was represented by a magical portal) are no longer needs. However, a portal system can be used to link between shards/servers/worlds/whatever. With this, we see the world going into multiple layers and an automatic level of detail to handel loads on the client side.
So from a far you ,may see a hundred characters that are only four hundred polys each, while when up close you see they can be up to 5000 or more.
While the V12 technology does look impressive and may be flexible, it wasn't intended for a MMOG(Wait for Tribes 3 though) and a lot of work would have to go into it; it would probably easier to find another engine that was made for what you are trying to do.
04/09/2001 (2:33 pm)
I think the best peice of advice would not be quoting technology sheets but say don't do a MMORPG.Not to crush anyones dreams here, but MMORPGS are too large to just for fun. I twould be nice that a small "Garage Games" company could do one, but its highly unlikely.
Possibily mentioned before; All MMOGs use zones. Call them what you want and try to cover them up, they are still zones.
The Turbine 2.0 engine(Asherons Call2) handels the world much like Asherons Call did. However, instead of covering up zones with portals, the "portal"(which was coincidently was represented by a magical portal) are no longer needs. However, a portal system can be used to link between shards/servers/worlds/whatever. With this, we see the world going into multiple layers and an automatic level of detail to handel loads on the client side.
So from a far you ,may see a hundred characters that are only four hundred polys each, while when up close you see they can be up to 5000 or more.
While the V12 technology does look impressive and may be flexible, it wasn't intended for a MMOG(Wait for Tribes 3 though) and a lot of work would have to go into it; it would probably easier to find another engine that was made for what you are trying to do.
#14
also Now i point you to www.glitchless.com where they are amking a MMORPG wiht a bigger land then AC can even imagine WITHOUT zones!
04/09/2001 (3:03 pm)
whatever, we want to make our own MMORPG for fun, and we will.also Now i point you to www.glitchless.com where they are amking a MMORPG wiht a bigger land then AC can even imagine WITHOUT zones!
#15
04/09/2001 (3:08 pm)
and other engines might be easier but we are all 16 year olds without a job so this is the best/cheapest solution out there. And don't be slamming us because we're 16 or we'll have to work harder on our game just to prove you wrong =P
#16
04/09/2001 (3:11 pm)
The EverQuest engine isn't suited for a MMORPG, what's your point? The engine has nothing to do with it. Feed the client the data, the engine will display it.
#17
The portals in AC really were just magic portals :) There was no need for them and at many stages during its development people suggested removing them altogether. The reason they remain is purely a design decision to facilitate transportation and help spread the population out. Also, magic portals are used to transport players to dungeons. Dungeons in AC are located in the extreme corners of the "map" where there is no land. This helps to distribute the population even more.
If they wanted to, you could enter every dungeon by going into a cave and going down some stairs. In fact, some dungeons are like this, such as the Lugian Outpost near Qalabar. The decision to use portals for dungeons was not based on a need for zoning, but for population distribution and the client rendering technology.
As for "crushing everyone's dreams," you should probably qualify your discouragement with "unless you really know what you're getting into." I look forward to people making enhancements to V12 for this purpose. If a team of people can add a MM server framework to the V12 core, other groups would be able to use that technology as a base for their games. From what I've read, GarageGames fully endorses people who want to do this sort of thing.
04/09/2001 (3:56 pm)
Matthew M is simply wrong about AC.The portals in AC really were just magic portals :) There was no need for them and at many stages during its development people suggested removing them altogether. The reason they remain is purely a design decision to facilitate transportation and help spread the population out. Also, magic portals are used to transport players to dungeons. Dungeons in AC are located in the extreme corners of the "map" where there is no land. This helps to distribute the population even more.
If they wanted to, you could enter every dungeon by going into a cave and going down some stairs. In fact, some dungeons are like this, such as the Lugian Outpost near Qalabar. The decision to use portals for dungeons was not based on a need for zoning, but for population distribution and the client rendering technology.
As for "crushing everyone's dreams," you should probably qualify your discouragement with "unless you really know what you're getting into." I look forward to people making enhancements to V12 for this purpose. If a team of people can add a MM server framework to the V12 core, other groups would be able to use that technology as a base for their games. From what I've read, GarageGames fully endorses people who want to do this sort of thing.
#18
Actually, Wes, I'm not. Yet the portals were there to cover up the zoning time. They may have near seamless outdoor environments(with sparse houses as cities) but to have their beautifully done dungeons they existed separate from the outdoor world and the magical portals were just a "portal/doorway" system to load the area.
And yes, I fully support anyone who wants to do what they can and how they want to. However, unrealistic pipe dreams should be realized and a more realistic and plausible goal should be made. And "You should really know what you are getting into" should have been added because it does back up what I just said and apologize for not mentioning anything like that in my original comments.
04/09/2001 (4:59 pm)
And it is a fact that "Dawn" and "Glitchless" is the biggest joke and insult to game developers everywhere. Sorry, but I am truly disgusted at the sight of them. And no, I am not bashing you, nor am I bashing you because your 16; its just there are goals and then there are dillusions.Actually, Wes, I'm not. Yet the portals were there to cover up the zoning time. They may have near seamless outdoor environments(with sparse houses as cities) but to have their beautifully done dungeons they existed separate from the outdoor world and the magical portals were just a "portal/doorway" system to load the area.
And yes, I fully support anyone who wants to do what they can and how they want to. However, unrealistic pipe dreams should be realized and a more realistic and plausible goal should be made. And "You should really know what you are getting into" should have been added because it does back up what I just said and apologize for not mentioning anything like that in my original comments.
#19
Making one large map is nothing if you think about it! Basically all zoning does is allow you to split the processing job, the job of processing players (not the map), between more than one computer/application. Think about it, a good server shouldn't know about the map (well, I cna't really explain it, it shouldn't touch a map too much if your doing server side collision detecting). Basically the job of the server is to keep track of npcs, players, and objects on the server which only requires x,y,z and directional coordinates.
p.s. I sortof see how Dawn can be a joke, but if it is, well damn a lot of people fell for it. there are a lot of contradictions. but whatever, once i get this engine i'll be workin on my own :)
04/09/2001 (5:43 pm)
Actully making a server for MM is not that hard (well i mean taking the V12 netcode and converting it to MM) because all you really have to do is to lessen the data that's being sent (or use compression). Making one large map is nothing if you think about it! Basically all zoning does is allow you to split the processing job, the job of processing players (not the map), between more than one computer/application. Think about it, a good server shouldn't know about the map (well, I cna't really explain it, it shouldn't touch a map too much if your doing server side collision detecting). Basically the job of the server is to keep track of npcs, players, and objects on the server which only requires x,y,z and directional coordinates.
p.s. I sortof see how Dawn can be a joke, but if it is, well damn a lot of people fell for it. there are a lot of contradictions. but whatever, once i get this engine i'll be workin on my own :)
#20
04/09/2001 (6:14 pm)
Then again, I am not a program, so maybe it is as simple as that.
Adam Yeager
-=[DocBaily]=-
"I'm out like a fat kid in dodge ball."
-baily