Welcome!

By registering with us, you'll be able to discuss, share and private message with other members of our community.

SignUp Now!

@EVAL's internal precision?

May
12,846
164
How many decimal places doe @EVAL use internally?

I mean, if I use @EVAL[w+x+y+z=2], does it add four precision-2 numbers or does it use higher precision and round the result to 2 places?
 
None of the above. @EVAL uses BCD (except for a handful of integer-only functions that use 64-bit integers).
I sorta knew that, but don't BCDs have some precision limit? And, if so, is it always the same?
 
I should also have asked ... it is correct, then, that a precision spec, like "=5" affects only the output, and not the calculation itself?
 
The precision limit for @EVAL is 10,000 decimal places. You set the precision to use in OPTION or in @EVAL.
Same question about that ... does it affect only the output and not the calculation itself?
 
Another, mostly unrelated question about @EVAL ... in a plugin, are these equivalent. Paraphrasing a little,

1. ExpandVariables(L"%@EVAL[expression]", 1)

2. ExpandVariables(expression, 1);
Evaluate(expression);
 

Similar threads

Back
Top