F64 (aka double) precision problem.

#1 If you do:

F64 a=1,b=2;
F64 foo = a+b;

and it breaks, it's probably due to either a compiler/hw bug (unlikely) or IEEE spec compliance (more likely). What are the values you are adding? It's way easy to lose precision on floating point numbers, even doubles.

#2 F64 B = -1519.4020996039750;
F64 JD = 2455304.0;
JD = B + JD;

results in JD = 2453784.5000000000

JD = 2455304.0 -1519.4020996039750;

results in JD = 2453784.5979003962

If I do exactly the same thing in TSE, on the same machine, with the same compiler, they both result in 2453784.5979003962. This should rule out both hardware/compiler bug and IEEE spec compliance.

#3 This sounds like the generic TorqueScript number of digits to display issue. IIRC, TS doesn't output more than a certain number of digits due to a string formatting parameter.

#4 I'm looking at this in the debugger, actually. This is wrong before it ever hits output.

#5 Hi Brett -

i tried your exact code in my TGE 1.3 under visual studio 8 and gives the correct results, even all the way out to script.

edit: vs8.

#6 Floating point precision is fleeting and ephemeral. It can be order of operation dependent and also limited by rounding modes, bit accuracy in the hardware, and so forth.

I notice in your "check" section that you swap the order of ops - does changing that affect anything?

#7 No, it doesn't, unfortunately. Nor does any sort of typecasting, or using += instead. Or anything else I've tried.

#8 Further update on my debugging tests:
It appears that at first startup, from placing the test code above into other places, it works fine during startup of the program, but once the startup screen appears, the accuracy drops and it starts rounding.

I am not, however, able to achieve the same result in a clean install of 1.4. Does anybody know how I could accidentally be screwing up the accuracy of doubles for the entire system?

#9 That is mighty weird.

try this:

[b]in engine somewhere:[/b]
consoleFunction(testF64, const char*, 1, 1, "" )
{
   F64 B = -1519.4020996039750;
   F64 JD = 2455304.0;
   JD = B + JD;

   Con::printf("JD is %f", JD);

   const char* ret = Con::getReturnBuffer(128);
   dSprintf(ret, "%f", JD);
   return ret;
}

[b]from the console during runtime:[/b]
echo(testF64());

#10 Double check your compiler and link settings? Perhaps you accidentally changed a setting in there that would effect precision?

#11 Results in "JD is 2453784.500000"

#12 Wild. what do you get with F32s ? with just plain doubles ? (should be the same as F64)

i've never heard of a float rounding to quarters or halves like that.
as ben said float rounding can definitely result in some unexpected numbers,
but this just doesn't seem to be in the ballpark for a rounding issue to me.

and you see the same value whether in printf or when you inspect the value w/ the debugger ?

what compiler/platform ?

#13 With F32s, accuracy drops down to where it may as well be an Int. Doubles are same as F64. Neither have I, and I wish I knew what I'd done to screw around with the precision of doubles for the entire system.

I'm using VC++ .net, on an XP Pro box, and yeah, debug's -exactly the same- as output to console.

#14 Your floating point precision is getting reset to 24 bits:

_controlfp(_PC_24, MCW_PC);

Possibly by some weird DLL linkage?

#15 Try this:

#include

_controlfp(_PC_64, MCW_PC);
double B = -1519.4020996039750;
double JD = 2455304.0;
JD = B + JD;

after whereever you have the code that generates the rounded value, and compare.

#16 You, sir, are now my own personal superhero. It worked like a charm. I will NEVER forget that function. Never in my life.

#17 As has been said elsewhere (guess I'm a bit of a naysayer :) - algorithms that rely on numerical precision in floats and can't deal w/ inaccuracies on a single, will not be fixed by going to double or doing other rounding changes.

You should design assuming error, and lots of it - that way will give you a much more robust situation.

If you need precision, don't get it by using floats - I'd highly recommend a fixed point notation, as that will allow you to ensure proper behavior across all hardware. For instance, if you want to support large worlds, using 128bit fixed point numbers for your position will give you precision at the mm level out to Pluto. All floating point representations will give varying accuracy across that range, and result in odd behavior at one point or another.

You _can_ work around it (by setting FPU control states and doing other clever hings), but it's better not to if at all possible - and of course, fixed point numbers work great on all hardware, including consoles, some of which have very limited FP capabilities.

#18 Ben -
very true, floats and doubles are subject to significant numerical inaccuracy,
but Brett was seeing an error of 0.1 in 2,455,304.

that's an enormous error even for floating-point.

i think it's reasonable to count on a much higher degree of accuracy from a double.

#19 Sure - but if you can't deal gracefully when you do lose precision, you're in trouble.

om Forsyth has a good article on some of the issues here.

I'm not saying to never use doubles, and obviously in this case there was some issue that was not making them function to spec. But I definitely sided with Tom and his OffendOMatic rule here. Sorry. :)

#20 Heh. i still think Brett was bitten sort of unreasonably.

Forsyth's point about dealing with two objects who are close to each other but far from the origin is very good.

also i would add to his rule ".. and any time you test floats for equality, you really, really haven't understood the algorithm."