@EVAL's internal precision?

[vefatica]

How many decimal places doe @EVAL use internally?

I mean, if I use @EVAL[w+x+y+z=2], does it add four precision-2 numbers or does it use higher precision and round the result to 2 places?

 

[rconn]

How many decimal places doe @EVAL use internally?

I mean, if I use @EVAL[w+x+y+z=2], does it add four precision-2 numbers or does it use higher precision and round the result to 2 places?

None of the above. @EVAL uses BCD (except for a handful of integer-only functions that use 64-bit integers).

 

[vefatica]

None of the above. @EVAL uses BCD (except for a handful of integer-only functions that use 64-bit integers).

I sorta knew that, but don't BCDs have some precision limit? And, if so, is it always the same?

 

[rconn]

The precision limit for @EVAL is 10,000 decimal places. You set the precision to use in OPTION or in @EVAL.

 

[vefatica]

I should also have asked ... it is correct, then, that a precision spec, like "=5" affects only the output, and not the calculation itself?

 

[vefatica]

The precision limit for @EVAL is 10,000 decimal places. You set the precision to use in OPTION or in @EVAL.

Same question about that ... does it affect only the output and not the calculation itself?

 

[vefatica]

Another, mostly unrelated question about @EVAL ... in a plugin, are these equivalent. Paraphrasing a little,

1. ExpandVariables(L"%@EVAL[expression]", 1)

2. ExpandVariables(expression, 1);
Evaluate(expression);

 

[rconn]

Same question about that ... does it affect only the output and not the calculation itself?

Precision is only for output. Internally @EVAL uses whatever precision is necessary for the calculation.

 

[rconn]

Another, mostly unrelated question about @EVAL ... in a plugin, are these equivalent. Paraphrasing a little,

1. ExpandVariables(L"%@EVAL[expression]", 1)

2. ExpandVariables(expression, 1);
Evaluate(expression);

Yes.

 

[vefatica]

Thanks!