1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

@EVAL's internal precision?

Discussion in 'Support' started by vefatica, Aug 16, 2017.

  1. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,888
    Likes Received:
    30
    How many decimal places doe @EVAL use internally?

    I mean, if I use @EVAL[w+x+y+z=2], does it add four precision-2 numbers or does it use higher precision and round the result to 2 places?
     
  2. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,809
    Likes Received:
    82
    None of the above. @EVAL uses BCD (except for a handful of integer-only functions that use 64-bit integers).
     
  3. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,888
    Likes Received:
    30
    I sorta knew that, but don't BCDs have some precision limit? And, if so, is it always the same?
     
  4. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,809
    Likes Received:
    82
    The precision limit for @EVAL is 10,000 decimal places. You set the precision to use in OPTION or in @EVAL.
     
  5. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,888
    Likes Received:
    30
    I should also have asked ... it is correct, then, that a precision spec, like "=5" affects only the output, and not the calculation itself?
     
  6. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,888
    Likes Received:
    30
    Same question about that ... does it affect only the output and not the calculation itself?
     
  7. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,888
    Likes Received:
    30
    Another, mostly unrelated question about @EVAL ... in a plugin, are these equivalent. Paraphrasing a little,

    1. ExpandVariables(L"%@EVAL[expression]", 1)

    2. ExpandVariables(expression, 1);
    Evaluate(expression);
     
  8. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,809
    Likes Received:
    82
    Precision is only for output. Internally @EVAL uses whatever precision is necessary for the calculation.
     
  9. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,809
    Likes Received:
    82
    Yes.
     
  10. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,888
    Likes Received:
    30
    Thanks!
     

Share This Page