Vehicles behave oddly at high mission coordinates -- 10,000

#1 As a followup, this seems to affect other object types as well. Players also lose position resolution at extremely high coords, however it doesn't have the same effect on them (I assume this is because they don't do the same types of intense physics calcs). I was under the impression that F32's could handle more than 6 decimal places, but that seems to be the limit here; you can either have 0.00001 or 100,000.

#2 Right... this is a precision issue. Remember that Torque was based around characters on a terrain where a 4KM landscape is considered huge. For your needs of a larger space environment with vehicles you should consider a larger unit size... like 1 unit = 10 meters. If you have alot of existing art, you might be able to scale most of it down and still have it work.

#3 The issue you might be having has to do with MoveManager and struct Move.

From my reading of the code, it only uses 6 bits when it reads/writes positional coordinates into the stream. You might need to change that to write more precision into the stream. Oddly, it uses 16 bits for yaw, pitch, and roll.

Take a look at Move::pack and Move::unpack in "gameConnectionsMoves.cc".

#4 Tom: The problem is that changing the scale to 1/10th doesn't actually help in this case. The amount of available precision remains unchanged, so when you drop to only 0.01 precision, the "jittering" effect will appear the same as it did at 0.1 precision at 1-to-1 scale because your models are 1/10th the size.

I suppose this does give you a larger area in which there is no jittering at all, though..

Tim: I'll take a look at that. The available precision does seems more limited than just the limits which would be created by F32's. Also, the rotation values are always high res, which is consistant.

EDIT: Took a look, but as I was doing the test I realized that the Move object is an input, and isn't directly used to move the object in the scene. It represents the input during the last.. however often a move comes in, hence why it only needs 6 bits to handle the position values; there's a limit to how much you can try to move an object in one update. The move x/y/z values are generally 0-1, with 1 being "move as fast as you freaking can" or "holding the move forward button down."

#5 @Henry - Right... with that change you only need to move 1,000 torque units to move 10,000 game world units. If tune down the velocities on your vehicles and scale down your art it will feel the same as before, but without precision issues.

#6 Ah... I didn't actually look at anything in the debugger. My bad.

I would have named those members "velx, vely, velz" in that case.

But, in retrospect, that's what the would have to be...

#7 Tom: The problem with this solution is that the shaking effect always exists. Even at 1000m, there's enough shaking that if your vehicles were 1/10th scale, it would already by just as visually unreasonable as it is at normal scale at a 10,000 coordinate.

I actually tried the 1/10th vehicle, just to see what the effect was like. Controlling a vehicle at that scale reveals an inherent precision issue with rigid which is in effect even at the lowest possible coordinates. The shaking never actually goes away, it just gets more and more difficult to notice relative to your scale as you get closer to 0.

#8 This happens with players too. I was experimenting and even the weapon the player holds twitches. How can I fix this without scaling my objects ( I have alot of stuff and that would take ages)?

#9 1) Adam: Unfortunately, there is no way to fix this at the moment. The problem seems to be the use of 32-bit floating points throughout the code for handling position; Floats (F32 in TGE) aren't going to give you more than 6-decimal accuracy, ever. That said, scaling your content, as everyone repeatedly suggests, will not help you; the relative accuracy of the position math will be exactly the same.

It seems that when the engine was developed, there was no need for accuracy beyond this; The environments from Tribes 2 were never larger than the size of one default-scale terrain page (2 x 2km). While this position inaccuracy does arise after coordinates higher than 1000, it's relatively insignificant until you go past 10,000, and doesn't become a serious problem until after 100,000.

Converting the entire engine to 64-bit floating point precision would be an unpleasant task, to say the least. Most of the physics/position math relies on math Classes like MatrixF (matrix float) and Point3F (3D vector float, again).

Just blindly converting the entire engine to double would also be a waste of memory; this level of accuracy is only needed for position variables, not rotation or anything else that's stored as a floating point in the engine. Converting only the actual storage variables in the base classes wouldn't solve the problem, as the math for position determination would still be handled with lots of F32's, Point3F's, and so on.

Now, I generally know my way around graphics, physics, game engines, that sort of stuff, but I'm not so well versed with the more in-depth nuts and bolts of programming, so I don't, for example, know if it would be possible to simply edit the basic platform support to force the compiler to make floats into doubles or something. I'm sure there's someone in these forums who knows a lot more about this than I do, but most Torque users don't really care about huge environments.

2) I think it's possible that this has something to do with the collision solver problems everyone's been trying to pin down for, oh, 3-4 years. I spent a long time on my current project "repairing" the Rigid class so that collision vs. other Rigid objects would work properly, and in the end I was forced to introduce ugly hacks to keep objects from falling through each other or ending up in unsolvable situations. The results look pretty good, but it's still theoretically possible to end up inside another object. Perhaps this lack of floating point accuracy is to blame; most of the impulse calculations done in Rigid produce extremely small values, and I suppose there could be a lot of inaccuracy building up here. Once again, TGE originally wasn't designed to handle lots of physics objects interacting, so this has never been a high priority issue.

#10 Thank you Henry

#11 I am looking at using TGEA for some space based maps, Henry did you every figure this out? Has anyone at GG gotten around to looking at this? I would assume that this would be a problem for anyone who has a map large enough to encounter the precision problem, space based or not.

#12 Converting to double precision won't actually "solve" the issue, it will just push it further back. There will always be issues with using finite numbers of bits to represent real numbers.

#13 I've encountered this situation as well while developing a space based game. With my game
design having a space mission that goes from -10,000 -10,000 to 10,000 10,000 gives plenty
of space for a solar system. When the client leaves that solar system they trigger the loading of
the next region of space contained in a separate mission. I was trying to do the space part of
the game all in one mission, but quickly found that wasn't possible. Then discovered that since
the space missions don't use terrain, and thus don't use a lighting file, they load very quickly
which was the original concern over having multiple space mission files in the first place.

The idea of scaling IS very helpful. The idea is not to lessen the jitter by scaling your vehicle, but
to give your vehicle more space to fly in by scaling everything down, including your speed. So at
normal scale if it takes 90 seconds for you to fly from one side of your mission to the other, if you
scale everything down (including your velocity) to 1/4th then it will take 360 seconds, and you have
a lot more "space" to fly in. (at least it looks that way to the client)

My first few missions have been prototyped and it's got a nice feel to it.

@Pat - Pushing it back would be fantastic. I would be very happy if the jitter didn't start until 100,000
or dare I say it, 1,000,000. The potential for space and flight games would be enormous if we could get
100,000.

#14 Right, but you are doubling the storage and not actually solving the problem, Check this out:
http://home.comcast.net/~tom_forsyth/blog.wiki.html#%5B%5BA%20matter%20of%20precision%5D%5D

Article on Tom Forsyth's blog. His blog is very worth putting in your RSS reader. He's actually moved on from RAD to Intel to work on a very interesting project, Larabee.

#15 Interesting read. It's quite true that using integers is the only way to be absolutely accurate in your physics calculations, as floats of any size will have significant accuracy decay at every step of the process. I was far too lazy to actually get into the guts and redo the position system in TGE, so I came up with something a little more absurd.

I've built a surprisingly functional solution involving "pages" of world space. In single player, this can be as simple as using triggers to create the "pacman" effect (ie, warping to the other side of the world when you reach the edge) and then adjusting the position of all the other objects in the scene accordingly. The effect works seamlessly, unless you are using a third person camera with any lag.

Obviously, it gets a little more complex in a multiplayer system. You'll need to store each object's x/y/z page in addition to its actual position within your 20k/20k/20k (-10,000 to 10,000) world box. You're still not done, because every object in the level now exists on top of every other object, since nothing actually ever goes beyond 10k in Torque coords. While everything might be inside the 10k box on the server, on each client the object positions will be adjusted based on their page ID's.

In other words, if you are Player A, at (0,0,0) in page (100,0,0), and Player B is at (100,0,0) in page (102,0,0), then on your client, your control object will be placed at (0,0,0) while Player B will be rendered at (40100,0,0). Why 40,100? The page offset between the objects is 2, the world size is 20k, and B is also 100 units X+ from its page origin.

Adjustments must also be made to the types of interaction you'll be expecting to handle. In my case, since this is for a space sim, that's basically just a matter of collision and projectile collision. Objects in different pages are not allowed to collide with each other (err, there are some "border" cases you need to consider as well, but that's less important), so even though they may occupy the same space on the server, they won't interfere with each other unless they're actually on the same page.

As for rendering, remember that you're not really going to have more than a 1,000,000 unit visibility range. This is way more than sufficient for ships and space stations, but real-size planets are still very large at 1,000,000 units, even if you're using a 1/10th scale. I dealt with this by having the client render fake, appropriately placed and scaled planet objects. It's important that the render order for these objects be set up correctly, as you want all distant planets to be rendered behind everything that's actually within you 1mil range, but you also want them to properly render behind each other depending on their distances (ie, for a moon which orbits and periodically moves behind a planet).

Finally, the same issues of floating point numbers apply to velocities in the physics system. You can't get an object to travel much faster than 1,000,000 m/s (or 10,000,000 m/s in a 1/10th scale), which sounds like a lot, but really isn't in true-scale space travel. At 10mil m/s, it'd take nearly 4 hours to travel from Earth to Mars, and your players won't appreciate that unless you're making a true space sim. I haven't tried to do anything with this yet, but I'll likely get around the limit here by having a "warp" mode which basically just increments the object's page coordinate and completely ignores the physics engine, allowing you to "travel" at incredible speeds.

This is all very "work in progress" on my end, but I don't see why others couldn't set up the same system relatively easily, especially the simple single player version (which requires scripted triggers only). It also seems like this could be adapted to work on land, though the ability to move the Atlas terrain around with no lag becomes critical here, and I've yet to even try that.

[edit]A final little note: Remember that "true scale" isn't actually very much fun by default. It's extremely cool to play with when you're flying around a planet with some moons, but actually traveling from Earth to Mars is uninteresting. For most of the trip, your destination is a dot, and players won't be able to navigate at all without a solid GUI for that purpose, especially if you start implementing proper orbits and use realistic space flight physics. Used correctly, it can be a really cool system, just don't think that realism equates to fun in this case.

#16 Very true. A 4 hour flight in a space game would be very boring for just about anyone.
What I'm going for is not ultra-realism, nor is it the cheap look of some old arcade games.
There's a happy medium in there where you can make it look and feel good, and still be fun.
With what I have done so far I think I've found that balance, at least for my purposes. It may
only take me 30 seconds to fly between Earth and Mars, but if it looks cool and is still fun,
then I've met my objectives.

I'm ok with doubling the amount of storage needed for a few variables. That shouldn't have
a huge impact on the system just for the movement of players and objects, as long as there
aren't to many objects. (so much for the asteroid field..)

#17 The Orbiter space flight simulator uses 64 bit floating point for all states (in meters) and calculations and they have similar problems as I recall bug reports, but these happen noticibly (in single player even) way out around the orbit of Neptune, IIRC, so increasing the precision would be a lot of work but can take you a long ways.

I helped build a trajectory optimizing program for NASA (OTIS version 3) where we attacked this sort of problem both by using high precision and also by changing the reference point of each vehicle state to the nearest gravitational body. This kept the numbers as small as possible while calculating. For example, when you do a hyperbolic pass around Venus the problem setup commands a change of coordinates to reference all states relative to Venus while in that mission phase. If much of your action tends to take place near "planets" or other points of interest you could use this kind of technique. This is roughly equivelent to adding a few more bits of precision, but the additional state info (which reference point is being used) is kept track of in new variables so it might be more convenient, or easier to "tack onto" the existing system.

#18 This is good info, Matthew.

(Warning: Speculation)
This problem seems similar to the way 3d transforms get used. Object->World->Camera->Screen. It seems like the way you could easily do physics is go Object->World-Reference for all objects involved. If you had assurances of a Reference point (gravitational bodies or what have you) at regular intervals, you could drop it down to 32-bits pretty easily and that would make the math a bunch faster.