TGEA for Seamless and Streaming terrains

#1 You're using terminology that is a bit confusing, which is probably why you're getting equally confusing responses.

Seamless terrain and streaming terrain are totally different. TGEA streams terrain from the harddrive for its Atlas terrain system, and for this, you can have a mission as big as the Earth itself in scale with the only limitation being the ability of the client hardware to stream it from the HD to the application.

Seamless terrain, on the other hand, involves multiple terrains with no seams between them, which is accomplished best using Mega Terrains (which then have seams, go figure).

What you're actually talking about is seamless zoning, which has nothing to do with terrain, but rather is a network issue. When zoning occurs, your player is essentially handed off to another server, dropped into a new mission area (zone), and then communicates with the new server for all interactions until it zones again and gets handed off to another server. When made seamless, the player can be handed off to a server without the loading screen or (hopefully) any perceptible interruption in service.

TGEA does not do this out of the box, and it is very hard to do, period. There may be an engine out there that accomplishes this, but I have not heard of it (I also have not really looked in a long while). What you want to do is Google for it, and doing that myself right now:

-C4 does not have it but plans it in a "future version".
-MMORPGMaker forum had a thread from '06 about "future plans".
-Irrlicht seems to have some code on their wiki that demos loading blocks of land, but does not seam to cover server handoff.
-Unity has an article about using it as a client to seamlessly zone between server cluster members, but the site seems down and that was all I got from the Google preview.

In any event, it's hard, and most released MMOs don't have it either. It's also not an indicator of success, as EVE Online can attest to (you didn't think those jump sequences were for anything other than covering up a loading screen, did you?).

But it can be done.

#2 Thanks Ted, you're right I am getting a bit mixed with the terminology, it doesn't help that every engine has their own terms for zone, mission, level as well! - but what you said is pretty much what I meant anyway.

Is streaming terrain the equivalent of paging?

Multiplayer aside, Mega Terrains sounds like it would do the trick - how does this work for lighting though? From my experience TGEA has a bit of loading time at the start to generate the lighting, does this work with seamless terrains since it can't realistically generate the light for all of your terrains at the start.

Or is there a substantial limit on Mega Terrains (compared to Atlas), so they simply light them all at the start anyway?

They must have something pretty impressive up their sleeves for Torque 3D if they are dropping Atlas. It's a shame it isn't Voxel though :(

#3 well that i know of and tested atlas cant have shadows or anything wile maga terrains you can. atlas you cant edit only in a external program maga terrains you can edit in the engine. you can add as many mega terrains as you want i think the same with atlas only thing is you cant mix mega and atlas terrains. i tried that and all that happens is the mega loads fine just there is no lighting for the terrain i tried relight and it still don't work. you might be able to get both to work but it will not be easy i even tried adding another sun.

you can do all this what i said in the demo maybe you don't know how to edit stuff when you go in the game press f11 to get to the editor. there are 2 demos Barricade Demo and Forge Demo barricade has the new one and its all messed up you cant load a new mission and you cant put down mega terrains as its a atlas map at start. go with forage demo you can do anything with the mission it loads still you press f11 but it limits you there so just load new mission and you have your first mega terrain and you can load as many as you want at lest i think cant remember if i tried loading more or not. you can edit the one it give you thought to test things out your self. you cant seem to access character though in forage as i don't think its been rigged yet. in barricade you can i don't remember what button needed to press to access character though as i uninstalled that demo as cant do much in it decides put objects in it.

i hope this help you some more this new site i don't think your going to find any of that info i just said its to hard to find anything now.

#4 @Staples - Lighting a mission is only done the once when you first play a mission (and again if that mission has been changed) so it's not something that you'll go through all the time.

#5 ok i did some more tests with maga terrains i think what they was saying us get rid of a portion of the terrain as that is in the engine u can see the sky when i do that. as for walking in it and falling i have no clue as character don't work in that demo. you could always edit code to make it the way you want it though.

i also added another mega terrain and i got it to work fine so you can add as many of them you want. i also tested muti textures on a object today as well normal map and bump maps work fine well that is if i had them correct textures but they work fine in the demo. o and relight the mission when making another mega as the textures are bright if you don't.

#6

Quote:Is streaming terrain the equivalent of paging?

Not really. Mega Terrain was supposed to be a sort of way to get more terrain blocks into the game because of the need for paged terrain blocks, but paging basically works like loading new blocks from the engine and unloading others, whereas streaming is loading and unloading parts of one terrain.

Quote:Or is there a substantial limit on Mega Terrains (compared to Atlas), so they simply light them all at the start anyway?

Like Andy said, the mission lights at the beginning of the game and then reuses that unless something affecting the lightmap moves. Generally, this means DIFs and (I think?) static objects.

A note on lighting is that if you load a DIF object into the game during runtime, you need to execute a relight to make it appear normal (it will show up as completely black otherwise), and this slows the game almost to a stop for a few seconds.

If you need to load multiple Atlas terrains, you can, since someone here took six of them and stitched them together to make a spherical planet. I've heard of multiple Mega Terrains (like Brandon is talking about I think). The thing with that is that you'd take that approach if you absolutely needed the type of terrain that Mega Terrain provides over Atlas, which would otherwise be able to supply more than enough terrain at better performance. They do have different feature sets though, that would induce some to use one terrain or the other.

Quote:They must have something pretty impressive up their sleeves for Torque 3D if they are dropping Atlas.

Didn't know they were dropping Atlas. Whatever they replace it with, hopefully it has all of Atlas' strengths and the ability to be manipulated in the editor. Making heightfields is one thing, but when they're that big, you'll definitely want to go in and make some changes here and there without regenerating the terrain every time.

As for voxels, I can see some limitations on it, or performance issues, especially with large terrains. I mean, sure, it's detailed and can look super-nice (especially with volume-based shaders), but it seams to me you can get almost as nice with the ability to import arbitrary terrain meshes (which have their own problems).

Anyway, hope this helps.

#7 well i did read them getting rid of atlas terrain in the t3d as well. in return i thought they said they going to make mega terrains larger and have them use less memory. i could be wrong as its been some days when i read that its in the forum the popular one with over 160 posts in it i not reading though all those again.

i don't think they adding a terrain to replace atlas ones but they are making mega terrains better. they going to be making editors better they said and be adding more but they not really saying much. :P

i would like to know what they adding in it i think they will tell us once they know what they are adding in it as well. they have not yet made up there minds at whats being in. some stuff is already made up but this is why there is a topic for you to post in what you would like to see in t3d they want to give us a engine we can all use. they want to make the price change well worth it they even said all updates will have more in it to make upgrading well worth are wile.

i rather like this t3d and i don't mind the cost ether as if you compare the price of t3d once it has a price on it. if you look at these engines that cost 100ks and mills i don't mine a little higher price for a engine. it might affect you guys though but not me i will save money to get what i want. a good engine is well worth it to me at lest what i have planed my fist game might not be perfect but i hope the second one i can do one of the larger projects i want to do.

i even did some collage to see what its like to try to get a aa and ba degree it would take to long to get it and i don't think its worth it. i had fun wile i was in collage but all i wanted to do is just take classes to learn more on stuff but you have to take this class in order to get in this class and so on so forth just to get classes you want to take it was not worth my time its ok my job is fine still its just not a gaming company. i will be doing all this on my spare time might even pay people to get it going faster but i do want a game a can be happy about finishing and after that hope many more.

#8 I believe that what they said on the new terrain was that it could support larger sizes and rendered faster than the previous renditions. They alluded to some other changes too but nothing was really mentioned that I have seen just yet.

Regarding Atlas, it can have shadows. A lot of areas in the engine didn't really support atlas out of the box though. I don't believe decals work on it. Some of the vehicle code needs some slight tweaking to make things work properly in atlas, etc. I'm a bit sad to see its going away, but if they deliver some great looking and high performing megaterrains, that will be forgotten. The main gripe with the current megaterrains is performance. Though it should be noted that if you reduce the visible distance the performance goes up significantly.

#9 EVE Online,well, in fact jump sequences are not only means of loading.
Much MORE data is loading when you entering grid(either via warp or via station undock) and if grid is filled with ships(and it WILL be in large battle), game becomes very interesting to play. Special skills needed -:)

I don't want to have SUCH effect in T3D(AFAIK TGEA's server doesn't support multi-server mode anyway so if it will lag due to this effect - it will lag everywhere).

p.s.Multiverse for example DOES support multi-server load balancing
(and on more Torque side,from published diagrams it looks like Minions of Mirth(made on TGE with replaced network layer)) is support somethink like this too).

p.p.s.As for T3D price goes up:well, if it goes up(counting discount for existing owners of both TGEA and TGE,even TGE 1.4.2 -:)) more than current C4 price for example I will seriously consider what to use next.
And T3D is better HAVE something to support new price increase. Good documenation(INCLUDING C++-level documentation, not only script level) will be good, either workable my current Torsion or integrated script debugger will be good too, same-for integrated world editiors.

#10 @Dmitriy: Regarding Minions of Mirth, they didn't really replace the networking layer. They improved it in a few areas, and that source code is available at www.mmoworkshop.com so you can see what they did. Some of those changes include lowering the tick rate and dynamic datablocks. There are a few things it does in addition to the Torque Networking, there's a lot of Python and Twisted Matrix code, seperate server processes in Python for the Character Server, Master Server, etc.

Minions of Mirth runs a large number of zones though on a single computer, and each of those zones is basically a separate instance of Torque. I don't think it actually load balances though. It can launch zones on demand though and shut them down when not in use.

#11 I have one doubt...How to Open dts & dsq files......which software is useful for this files.....can any one plz tell me that one....

#12 As for "Mega terrain versus Atlas", I think Atlas simple offers more than Mega Terrain(MT) if size is you main concern. Yes you can stitch more than 1 MT together to create a very large terrain but some very nasty technical problems can result.
Take this bit of info from the console.log from the T3D example that comes with TGEA 1.7.+.

Quote:
Allocating a 512 px clipmap for a 16384px source texture.
- 5 base clipstack entries, + 1 cap.
- Using approximately 7.980000MB of texture memory.
Allocating a 512 px clipmap for a 16384px source texture.
- 5 base clipstack entries, + 1 cap.
- Using approximately 7.980000MB of texture memory.
Allocating a 512 px clipmap for a 16384px source texture.
- 5 base clipstack entries, + 1 cap.
- Using approximately 7.980000MB of texture memory.
Allocating a 512 px clipmap for a 16384px source texture.
- 5 base clipstack entries, + 1 cap.
- Using approximately 7.980000MB of texture memory.

Now the important part is that a single MT is using 4 clipmaps that eat about 32 megs of memory, video memory I believe.
So if you start stitching MTs together you start to eat up memory at an alarming rate.
All a MT is is 4 legacy terrains very cleverly arranged, not a precise definition but close enough.
And because of this each MT has a resolution of 512 X 512 points or 4 256 X 256.
Now you can increase your squareSize(the distance between horizontal points) to make a MT larger in size but you still only have a 512 X 512 map of vertical points to work with.

Atlas on the other hand is more flexible in regards to size and detail.
A simple example to illustrate my point.

A MT with a squareSize of 8 will create a terrain 4096 X 4096 in size but only have 512 X 512 of horizontal data points to shape the terrain.

A Atlas terrain can be made at the same 4096 X 4096 size but can have a horizontal scale of .5 resulting in 8192 X 8192 horizontal data points.
If a horizontal scale(squareSize) of 8 is used like in the MT you end up with a 65536 X 65536 terrain.

MTs do have the in game terrain editor and can be "painted" in the editor as well and that is a definite plus but L3DT and Grome both can this and then export it to torque.
As far as editing goes I have to give it to MTs. Nothing can equal and actual in engine real time terrain editor for creating a gaming experience.

I personally like Atlas over MT.
I think that the potential of Atlas was never fully explored.

I don't know what GG has in store for terrains in Torque3D but if its MTs with real time lighting I will be disappointed.

I will be looking very close at Torque3D and if the new terrain system can't live up to the potential of the Atlas system then I will try and reincorporate it back into Torque3D as a resource or something.


@ Ted - Forgive my shameless self promotion.

And from the dark side.

#13

Quote:Does TGEA support streaming terrains (what I consider the ability to load the next mission/map without a loading screen)

This is what we are planning to do with Visions. TGEA does NOT support this out of the box, but it should be possible to modify the engine to support this with some pre-loading of the missions. Time will tell if we can actually manage this but this is our current plan.

As to loading multiple megaterrains into a single mission, we believe the maximum number of megaterrains we can load in a single mission without suffering too greatly on the performance hit is 4. So that's 4 megaterrains which equal a total of 16 terrain clips. We have heightmaps for 9 zones created, with many more planned, although we have only just started the actual terrain creation (and ran into a set-back when we discovered that the MT is importing the heightmaps backwards! so now I have to flip all our images horizontally and reimport the terrains and start over). So we have yet to see how well we are able to implement the seamless zoning plan. Please keep us in mind if you come across any resources for this! We would be very interested.

God bless you,
-Sparkling

EDIT 5-10-09: Further development is suggesting that by the time we get the terrain painted, the foliage placed and all the constructs in place we may only be able to use 1 terrain tile (which auto-splits into 4 pieces) per zone due to performance issues. Which is a little disappointing, to say the least. We have to deal with it though. The EULA on T3D is prohibitive to a volunteer development team. And I would really rather not change engines again. All along, in developing Visions, terrain has been our biggest issue.

#14 Hello.. I too am looking around at engines. Since I've played various MMO's and such, not as a gamer but more from a looking at technology standpoint, I have gotten pretty used to systems like the one found in Guild Wars..

While you sometimes have to wait when entering a portal or entrance to a new zone, you can see the zone that you are going to be going into. I like this concept but I wonder if it's possible to have set areas on a level that could do this. For example, Using the Legacy terrain, there are 4 sides. Maybe each of the four sides leads to another area. If this is the case, then once the player reaches a certain distance to the portal/entrance/pretty particle thing, you would go ahead and pre-load the next area so that the player wouldn't have to wait.

Another feature I like about Guild Wars is that instead of big patches, it will stream new and updated content in the background while you play the game. I think that is pretty cool as well since if you have a small mmo running but only add a few trees or something to an existing level, you only have to stream a few new models, images and the .mis file containing the new placement.

I could be wrong on some of this as I've never really studied any docs, that just what I've seen from playing the game awhile back.

Chris

#15 There was a guy who was selling some code which did that with a bunch of legacy terrains, tiled together and each could have their own texture and sky settings. Unfortunately he sold the code "exclusively" to about 5 people and noone could figure out who owned the code, or if he even owned the code to begin with. His english was bad and in the end I don't think anyone ever used the code for fear of it being stolen. It certainly can be done though. This BTW was with TGEA 1.03.

#16 "In any event, it's hard, and most released MMOs don't have it either. "

All commercial MMORPG's like World of Warcraft, LOTRO, Warhammer etc. have seamless zoning or what it's actually called --- a Dynamic Loading system. They have to otherwise you would experience a major stall when crossing over land patches because instead of loading everything in the next patch seamlessly in the background (as a Dynamic loading system does) it would have to be loaded into memory first.

There is no "plugin" for a seamless loading system that I know of. It's built by the developers of a game for their game as their developing. It's for massive worlds.. not small multiplayer type games on single small maps.

There are some engines out there that have the ground work for a Dyn Loading system.. Big World for example has something like that..Hero engine.. Abyssal Engine...

#17 "There are some engines out there that have the ground work for a Dyn Loading system.. Big World for example has something like that..Hero engine.. Abyssal Engine..."

I know developers on both BigWorld and Hero engines and they are $1,000,000 price tags. Sure, we would LOVE to be able to use them. But how many indies have $1 mil laying around? I wish it weren't so but it's just not going to happen. Only an angel investor could make it happen for us and that is slim to none.

Hero engine has a built in collaborative editing tool as well. That would be awesome.

Yes, we are using a massive terrain in our massively multiplayer game. It's the only way to create anything resembling historically accurate geographically relevant terrain. I am hoping we can find someone who can help us figure out seamless Dynamic Loading. In the meanwhile, we keep doing as much as we can in the time we have.

I hope if one of you discovers or creates a seamless loading resource that you would share it with the community. I'll keep watching this thread in that hope. :)

God bless you,
-Sparkling

PS - Happy Mother's Day to all the Moms out there!

#18 I want to make The Solar System as 1 giant level to scale.

Is this possible with TGEA?

#19 I don't think it's possible with almost any engine out there, out of the box. Have a read of this series of articles:
http://www.gamasutra.com/view/feature/2984/a_realtime_procedural_universe_.php

Also, creating a new topic would have been a fantastic idea.