we still need atlas terrain in T3D

#21 Well, they say that it's better to be silent and considered ignorant than to open one's mouth, and prove it, but since I don't understand what causes the precision issue, I have to ask: What is more stable 4096 w/ square size of 4 or 2048 with square size of 8? Or are they completely equivalent in terms of stability? Certainly the terrain with the smaller square size would look better, at least with the stuff I have been bringing in from L3DT it does.

#22 The terrain has nothing to do with the precision issue. If you place an object or have your player out at 10000+ coords, things start jittering. You can make a terrain as large as you want, its just anything thats on that terrain 10000 units or more from the origin that has a shaking issue.

#23 time for stupid questions, feel free to point out if they are...

1) i cant seem to find a setting that adjusts the height range a'la TGEA, i'd like to tweak this if possible.

2) is the terrain heavily integrated or is there some kind of wrapper, can i remove the terrain system and replace it (not necessarly me :p but functionality wise)?

3) if i use 4x 1024 heightmaps do i gain anything over using 1x 2048 heightmap, does this resolve the 8196 limit and/or does this resolve precision issues.

For what i'm planning i want several areas that are relatively small (in the open world large landscape sense of things) but was hoping for a single larger higher detailed world map, with rivers and such, larger squaresize really messses with this so finer control would be better for me.

With regard to some of the comments about filling areas id have to agree, however there are scenarios where i'd like nothing but a massive area of forest (again detailed so smaller squaresize).

the major issue seems not to size itself, as pointed out, larger squaresize makes this a possibility (btw is 8192 @squaresize 8 actually possible?) but in fact the detail levels, i love the river, but you really need to drop the squaresize to do it any real justice, and dependong on what your target audience is the 1m squaresize really does damage performance unless you have a smaller heightmap.

anyway enough rambling from me :)

#24 As was mentioned somewhere else, the precision problem could be addressed with a periodic re-centering of the terrain. I am not saying it would be easy, but it would probably be something that would have to be part of some paging* terrain system along with some kind of resource management that Manoel mentioned.

I would just like to add that I would really like to see something that would allow huge terrains to be used.

*edit: or "zoning", or whatever the proper term is.

#25 1) I think this has to be done when initially importing a heightmap.

2) Not my area but my guess is that its not too heavily woven into the engine so you could add your own terrain if you wanted.

3) Again, not really my area but I would think using 1 terrain block would have better performance than 4 of them.

#26 again i'm not entirely up to date with terminalogy of everything but theres terrains that only render what you see not so much occlusion as distance culling, this is how TGEA does it i think, and then you have cell based where it loads all 9 cells based on your position in the center cell and loads and unloads as you move into one cell from another (is this one paging?)

This seem similar to what T3D except we get cell LOD rendering, which is ok unless you have mountains and valleys, and it can quickly make distant mountains quite ugly sometimes as you see the LOD changing as you go

#27 One thing I noticed in TGEA with Atlas, you can view the entire thing, actually I had like 3 or 4 of them and they were all in view. Terrains on the other hand vanish into the digital ether if you are too far from them, but the atlas still shows up. So I had the idea of using atlas's for seeing the landmass at a distance but the nearby land would be the regular terrain. I could never get an adequate level of "up-close" detail in an Atlas due to the size I wanted to make it, but it looked good from a ways off. The terrain can then be placed and shaped to the atlas to provide a higher level of detail to the area the player is in. This helps provide immersion into a large game world by handling both near and far views of the terrain. So, I sincerely hope Atlas will be implemented in T3D. It seems as though just as you guys got Atlas to work great in TGEA you decide to give up on it. Don't get me wrong, I realize there's a lot of more important things to take care of first. I'd just hate to see a really good feature disappear for reasons hither to unknown.

#28 About floating point issues, if you did a paging map system, you could simply do a [blockid][x][y] add have block id be unique pr block, that way, each block would start from 0,0 and would never be too big.

so when you place objects, blockid is the important, and xy would be where on the block to place it.

There, issue gone :P

#29 I do think its fairly essential that we get a fix for the floating point precision issue, as it is a fairly big "Limitation". Especially for Flight Sims, and other vehicle games.

#30 Is GG going to do anything about this horrible limitation/show stopper?

Quote:The terrain has nothing to do with the precision issue. If you place an object or have your player out at 10000+ coords, things start jittering. You can make a terrain as large as you want, its just anything thats on that terrain 10000 units or more from the origin that has a shaking issue.

#31 but why? Not that it matters, but just for my personal education, can someone give a simple explanation of why this happens, the over 10,000 units jittery thing? Also I hear some people say 10,000 units is where it starts to have problems, but others say 15,000. Is there any more definite value?

#32 @Steve: Computers cannot have infinite precision. A floating point number has variable precision: the number of bits allocated for the "integer" and the "decimal" parts vary depending on the number being represented. The larger the number is, the less bits are used for the "decimal" part. Here's a nice explanation of it:
www.floatingorigin.com/

There's nothing to fix. If they fix it for your massive flight sim, it breaks my small ridge-racer-like racing game because now every object's matrix takes twice as much memory and is slower to compute because its using 64-bit floats. Also, those matrices will need to be downsampled to 32-bit before being feed in PhysX and before being sent into the GPU for rendering, so jittering would still occur.

Creating gigantic worlds is not a stock feature, and never was, and I don't think it will, since its obviously very game-specific, because its *not* simply a matter of bumping coordinate values to 64-bit (which is not truly possible because not every part of the process will be able to deal with 64-bit matrices, like the GPU). For that you need hierarchical coordinates.

There's a bunch of articles and tutorials about that on the net, but here's a summary: to get truly vast environments, your objects must give up on having absolute coordinates. Instead, each object only knows the zone/block it is at and its position is relative to that zone/block center. The zone/block coordinates are on a much smaller scale. The camera too must know in which block it is. When rendering, you calculate render coordinates for every object using the block the camera is located at as the center. It's also necessary to deal with moving objects from a block from another and when handling collisions between objects in different zones (the boundaries).

In Torque lingo, this means changes to the SceneGraph (rendering, binning, scene traversal), the SceneObjects (adding the block info), the Container functions (castRay, radiusSearch, etc) and probably collision code for the classes you're using (the working collision set stuff, buildConvex, getPolyList, etc).

If having a gigantic world that goes beyond floating point limitations is paramount for your project and you have zero means of implementing it yourself, I'd suggest you look for a solution where that's a key feature.

#33 Just using 64bit is no solution, thats just a workaround to push back the real solution to the problem to a later time (when 64bit will suffer the same problem again which will naturally take a while but it will happen again thats granted)


realistically, the only real solution is world compression & time compression, usage of camera zoom and a few other things if you want to have a pseudo massive world.
Especially flight and space games, the only games which have really such a far view distance without blocking, benefit from camera zoom and "compression"

#34 i was aiming to build at a minimum a 20kmx20km terrain (preffered 36x36km), now bareing in mind this +/- 10,000 unit limit, how can this be acheived?

@Marc, can you expand on time compression, usage of zoom?

I think i follow you to some extent, make all the props and models smaller, lower the view offset?

Surely increasing this limit, although increasing memory, would still be a good thing?

@Manoel, it wouldnt break your rigde-racer game, it would allow you to construct more detailed terrains, and have larger racing tracks and circuits, for the cost of a little bit more memory? that sounds worth it. =S

#35 @Manoel -- I do appreciate you taking the time out to share that explanation.

@Bloodknight -- It is my understanding that attempting to import any terrain larger than 4096 will crash the engine.

@Matt Langley -- Well, it may or may not be true that only a few teams need larger terrains. Maybe only a few need the really huge ones, but I think many teams would benefit from something larger than we have now. Therefore, I hope GG considers it. I think it would help Torque be a more well rounded engine. As it is now, one recurrent criticism of Torque I hear is that it is not a good engine for RPGs or MMOs. Of course you can never serve all the people all the time, ie, you simply cant have a large enough terrain. Bigger Terrains, RPG genre pack, and that better deal / better features on the T3d basic, and you'll at least have one happy customer :-) .

#36

Quote:I think i follow you to some extent, make all the props and models smaller, lower the view offset?

Yes I was going to say this The easiest solution is to adjust the scale of you objects "including you terrain" to say 1/3. If you need a world that is 32KM and everything goes crazy at say 15KM well I think the easiest solution would be to scale everything in the game down to one third the size it was. at this new scale you world not has a theoretical limit of 45KM before everything goes crazy.

Beside do you really need that much space I mean the best RPG of last year "fallout 3" was only 14 square miles. Just under 4x4 miles.

As for flight sims. Flight games like H.A.W.X. just use a smaller scale. Any game is doable you just have to find a balance that fits within the limits and suites you needs.

Using the default scale wanting to scale the world up to match "brute force approach" is not very realistic. And for the reasons discussed here is why it is never used even by the Big AAA studios.


The only solution that has been discussed that I think is even remotely viable here "other than scaling everything down" is Bo's approach.

#37 While many people are claiming that the precision is something that is only specific to a certain genre, it just leads me back to Tom's previous comment:

Quote:You can happily use a squareSize of 0.1 or 1000.... it just works.

This is the approach that I think should be taken on the matter. It makes sense to impose limits upon the user within the engine, within reason of what is feasible.

However, if there is a solution that can be put in place, that solution would create the situation where users would not have to worry about it. It would just work.

I think ultimately when building an engine that is designed to be multi-purpose the more things you can get to just work where the user doesn't have to know the quirks and nuances the better.

I believe it would require a lot of changes to come up with a new positional space. So I would say it's out of scope for T3D, but not a worthless endeavor.

#38 I think that people are missing he point thought when they complain that they cannot make the terrain big enough. Making the terrain bigger is a moot if you cannot viably use the portions of it beyond the precision limit.


I have been racking my brian to think of a game that has a bigger world that was T3D is capable of and I have yet to think of one. The exclusion to this would be a MMO or something but that is a completely different animal and thing have to be broken up anyway for load balancing purposes.

There is nothing that cannot be accomplished within these limits. you just have to be a little more creative.

#39 @Aldavision: a common racing game, console-style, would have no benefit of 64-bit coordinates. Also, video cards cannot use 64-bit coordinates consistently. You'd need to convert them to 32-bit before they could be used, and then problems would arise again.

To make it clearer: I don't think any game out in the market actually uses 64-bit coordinates. Why? because they don't fix the problem: you're wasting performance and memory to merely delay the problem. Games either dangle on the accuracy limit (where scaling things down by 0.1 like James suggested might do the trick) or are expansive enough to eventually hit 64-bit limits anyway.

Really, I stumbled upon a couple papers and thesis on this while google'ing on the issue. 64-bit is not the answer, and brings more problems than it solves.

@Steve: there are more technical explanations on the net, but I think I can explain it in simple terms:

With integers and fixed point you have fixed increments between numbers. Floating points use a variable increment that increases along the number. It's such that with very large values you lose the ability of storing fractions altogether. It's not a Torque problem, it's a computer problem. You can test it in Torque's console:

==>echo((10000 + 0.54) - 10000);
0.540039
==>echo((10000 + 0.55) - 10000);
0.549805
==>echo((10000 + 0.56) - 10000);
0.55957

At 10k, we can already see some inaccuracy. None of the fractions is accurately represented. But the difference seems small, no? Let's test with bigger numbers...

100K:

==>echo((100000 + 0.54) - 100000);
0.539063
==>echo((100000 + 0.55) - 100000);
0.546875
==>echo((100000 + 0.56) - 100000);
0.5625

1 million:

==>echo((1000000 + 0.54) - 1000000);
0.5625
==>echo((1000000 + 0.55) - 1000000);
0.5625
==>echo((1000000 + 0.56) - 1000000);
0.5625

Going 64-bit reduces the problem, that's true, but working with 64-bit coordinates is a challenge, since they are handled by multiple paths and not all of them might work with 64-bits. Also, this is not only position: coordinates must be part of the transform matrix, which is a 4x4 matrix. Meaning all transform operations (rotate, scale, etc) need to be performed at 64-bits. Collisions will also need to work with 64-bits, draining even more performance. And after all that you will still face loss of accuracy if you travel far away enough from the origin.

The only reliable solution is to not have world coordinates anymore, and thus never have to deal with massive numbers. But then, again, this is something *only* games with truly gigantic worlds need, which changes how coordinates fundamentally work in the engine (not a "on/off" switch by any stretch) and thus I'm completely against such thing being a standard feature.

Since this kind of request (huge/lifesized/infinite worlds) sparks once in a while, I wonder why nobody creafted a resource or kit to address the issue.

#40 @Manoel -- I understood your explanation and the link you sent the first time, but this makes it clearer, thanks. However it does bring something to mind. If we are dealing with floating point precision and number of digits, does this imply that any number from 10000 to 99999 would have the same precision since they could allocate the same number of digits to the decimal part? And, referring to your example above what magnitude of error can a game withstand before the problem becomes noticeable?

@Brett -- I'm not sure I understand what you are saying, but in regards to using square size of .1 or 1000 and it just works..um, it might still work, technically, but if you end up with a terrain that you cant put anything on because it will be jittery, then of what value is that it "just works"? Example: Terrain size 4096 x 25 square size. Aside from looking like crap, wont that mean 100,000 world units, and therefore the jitters? Or am I misunderstanding something?