Floating-point precision and why we can't get around it

I just wanted to put this idea, finally, to bed. Often, people ask how they can extend the range of their game world in Torque beyond the mythical 10km-from-origin limit. Often, they are told that they can scale down all their objects by, say, 10x, so the 10km limit becomes a 100km limit. While this seems like a great idea, it's sadly not the case.

When you scale down your distances and sizes, you make the tiny errors that exist in floating point numbers more visible. Imagine you're a human, and a small rock at your foot jumps a millimetre to the left. You don't notice, it's a tiny error. But now imagine you're an ant next to the rock. Suddenly, the rock moves half your body length!

Here are some helpful references from around the web:

• Floating-point arithmetic: issues and limitations (Note that Python uses 64-bit decimal values, while Torque uses only 32, so they push the issue back a bit. But it's still present)
• A real-time procedural universe part 3: matters of scale (Contains a lot of advice on how to get around the floating-point precision limit when rendering far-apart objects and so on. Final solution: use 64-bit doubles for object positions, 32-bit floats for everything else.)

For the rest of this post, I'm going to use the T2D console to demonstrate this happening. It's not essential reading. I just want everyone to understand: there's no free lunch when it comes to floating-point numbers. Scaling them doesn't work.

Okay?

Okay.

Unfortunately, floating-point numbers' greatest strength is also their weakness. While they can represent extremely large values as well as extremely small ones, they don't do well at mixing the two, and this is why errors start to turn up as you travel farther and farther from the world origin. Let's start simple, by manipulating a float in T2D's console. We'll add one millimetre at a time to an initial 10 metres displacement. Say, for example, you have a very slow-moving car.

==>$x = 10;
==>$x += 0.001;
10.0010004
==>$x += 0.001;
10.0020008
==>$x += 0.001;
10.0030012
==>$x += 0.001;
10.0040016

I've omitted all the echo() calls for brevity. The thing you should understand from this log is that floating-point numbers are never accurate. It's just often the case that we don't notice the inaccuracy. The errors, the very last digits, are on the order of micrometres, which you're never going to notice on-screen. Who cares if the engine is a micrometre off?

Let's try some larger numbers.

==>$x = 2000;
==>$x += 0.001;
2000.00098
==>$x += 0.001;
2000.00195
==>$x += 0.001;
2000.00293
==>$x += 0.001;
2000.00391
==>$x += 0.001;
2000.00488
==>$x += 0.001;
2000.00586
==>$x += 0.001;
2000.00684

After adding 7 millimetres, we achieve an actual displacement of 6.8 millimetres. That's not the end of the world, but you can imagine how this can build up. These errors manifest themselves as simulation instability as well as rendering artifacts - objects jittering and so on. Because these errors show up in relative calculations as well - for example, two objects close together but far from the origin:

==>echo(5000.018 - 5000.017);
0.0009765625

Still not too bad, but you can imagine this is a bit unsettling for detailed physics or collision detection calculations.

So, let's try and increase the accuracy by scaling down. We'll divide everything by 10, so 200 units is 2000m, the same value we were working with before. Now, to add one millimetre, we have to add 0.0001:

==>$x = 200;
==>$x += 0.0001;
200.000107
==>$x += 0.0001;
200.000214
==>$x += 0.0001;
200.00032
==>$x += 0.0001;
200.000427
==>$x += 0.0001;
200.000534
==>$x += 0.0001;
200.000641

So after adding 6mm, we end up with a 6.4mm displacement. Though the error is a smaller number than before (we're off by 0.00004 instead of by 0.0002), it looks bigger because everything around it is smaller. In fact, the visual error is worse than before! (That's just bad luck: in general, scaling down doesn't make you less accurate, but for these numbers, the errors happened to work out that way.)

The idea that you can get better floating-point resolution by scaling everything down is similar to the idea that you can give your television screen a higher resolution by sitting closer to it - it doesn't work that way. Sorry to harp on about this. It's one of the most persistent game engine myths I've seen, in other forums/engines as well as here, and I've wanted to go all Mythbusters on it for a while.

Oh, and don't think this is a Torque-specific problem. I just used the T2D console because I had it handy. This is a problem that happens to all applications that use floats.