issue with number precision in scripts - LOGGED

#1 Hi Rex,

I use bigger floating point values than that in my scripts and have no problems, so I did some testing. The problem seems to be with the echo function, in that it is rounding the numbers when displaying them. As far as I can tell the precision internally in Torquescript is ok, ie when assigning values to variables and performing arithmetic on them everything is being done ok.

From my tests I see that the echo function is rounding float values to the nearest integer when displaying numbers with more than 6 digits.

Annoying, I know, but echo is only really for outputting debug info to the console, but I agree that better precision would be more useful. If I get some time I'll look at the code and see what's going on, but it won't be until the weekend.

#2 It looks like it is more than an issue with echo. Here's what I just tried:

%val = 0;
for (%i = 1; %i <= 6283; %i++)
  %val += 159.137;

txtResult.text = %val;

In this case, "txtResult" is a guiTextCtrl that shows the result. The result should be 999857.771. What I am seeing is 999807. This indicates that the precision is lost internally. And if you bump up the 6283 to 6285 you have the result displayed in scientific notation.

I tried searching the forums and found a few posts that seem to indicate it may be an issue with how scanf is reading the values but no fixes were offered and i'm not sure where to look to find they were talking about.

#3 Sorry if I'm missing something here, but this seems like a standard float precision issue. Precision is always going to turn into a mess once your numbers get large... everything in the engine uses floats, which are 32-bit, and really don't handle huge numbers especially well.

Whether the scripting system is making the precision issues even worse somewhere internally, I can't say. You'd have to follow the code thread that actually does the work when you execute arithmetic in script and see what's really going on.

For your project, that apparently does a lot of large number calculations, you may need to look into exposing some basic script support for 64-bit floating point calculations and variables.

#4 If I remember well, there is a resource in the forums for 64-bit floating point calculations in torquescript.

Edit:
One from the Elenzil's Collection,
torquescript math routines for large integers.
and
64bit math library for TorqueScript

#5 Ah, good links. Just remember that your resulting answers are strings, and as such you basically can only use them to output in text form. You won't be able to use these resulting values in TorqueScript conditionals, numerical datablock/GUI element/whatever values, etc.

Just looking at that second resource, you could also throw in a function for greater than, and a quick "macro" to do == for floats as well, so you could at least use the results of these functions in conditional statements.

Didn't test these, but I'm pretty sure they'll do the trick (add them to the list of functions from that second resource):

ConsoleFunction(m64Greater, bool, 3, 3, "strings:(A,B)\n"
                "Returns true if (A > B)")
{
   F64 a = dAtof64(argv[1]);
   F64 b = dAtof64(argv[2]);

   return (a > b);
}

ConsoleFunction(m64GreaterEqual, bool, 3, 3, "strings:(A,B)\n"
                "Returns true if (A >= B)")
{
   F64 a = dAtof64(argv[1]);
   F64 b = dAtof64(argv[2]);

   return (a >= b);
}


ConsoleFunction(m64Equal, bool, 4, 4, "strings:(A,B,Tolerance)\n"
                "Returns true if A is within Tolerance of B (~==)")
{
   F64 a = dAtof64(argv[1]);
   F64 b = dAtof64(argv[2]);
   F64 tol = dAtoF64(argv[3]);
	
   return (mFabsD(a - b) <= tol);
}

If the equal function seems confusing... you supply the two variables to compare and the tolerance that you consider "close enough to be equal" in this calculation, since (as was the point of all this in the first place) floats can have small precision inconsistancies making it common that two values which should be equal will be just slightly off.

As the resource says, make sure you're passing strings (put the numbers in quotes) instead of numerical values.

#6 EDIT: Just saw the updated posts (#5&6) so please understand the following was written before I read those! Checking them out now. Thanks

That almost helped but still isn't correct. The precision is still off and it's not like I'm trying to go to ten decimal places. I'm still using the same test as earlier but sending the results to a guitxtctrl instead of using echo. I'm starting with "0" then adding "159.137" using strings as the m64Add requires. Even on the first loop the value being reported is 159.13699. As the loop continues the value is further from the number it should be. And it gets worse with the later iterations.

So I thought I would eliminate the decimal problem and start with the value multiplied by 1000. Using the number "159137" I start adding. A strange thing happens when we hit iteration 10. At this point we are adding our "159137" to the current sum which is "1432233" which should give us "1591370" but No! The result comes back as "159137". Hey, wait... that's the number we were trying to add! Gha!

Fine. Maybe dealing with a float for a large integer value isn't the way to go. So I took the m64Add function and created my own m64AddS

S64 dAtoS64(const char *str)
{
	S16 a=1;
	S16 b=2;
	S16 c=0;
	c=a+b;
	a = c-b;
   return (S64)atol(str);
}

ConsoleFunction(m64AddS, const char *, 3, 3, "(A,B)n"
                "Returns A+B")
{
	S64 a;
	S64 b;

	a=dAtoS64(argv[1]);
	b=dAtoS64(argv[2]);
	
	S64 res=a+b;

	char* buff=Con::getReturnBuffer(64);

	dSprintf(buff, 256, "%d", res);

	unPad(buff);

	return buff;
}

....and still run into the same issue! And trying to debug things at this point is a bit difficult. Visual Studio won't show the values for the vars when you hit a break point in the ConsoleFunction declarations. I checked on the function that converts from string to long and it seems to be doing the job correctly. The only thing i could find that i couldn't explain (or figure out) was something in the call stack about a dictionary object "Dictionary::Entry::setStringValue" in consoleInternal.cpp. It's either that or an override of the "+" as it relates to S64 but I couldn't find where that's handled in the code.

This is really a problem for the app that i am working on. About the only other thing I could think to do is build a "string math" lib but that would really slow things down. Any idea how to fix this?

#7 Logged: TQA-401

#8 I use a string math lib like you said to do this, I have it integrated with torque and it does the job for me.

We licensed a commercial math product for this part.