You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The internal value for IntegerInput, DecimalInput, and other derived inputs, is stored as a JavaScript number. In most (all?) implementations, numbers are stored as double-precision binary floating point values. However, integer values are stored Python-side as Python ints (or longs), which have exact integer precision, and infinite range (although Axiom has range limits. Also, decimal values are stored as Decimal values on the Python side, which have prec=28 in the default context. Thus, values lose precision when sent from the server to the client; and values entered by the user also lose precision compared with the server-side capability.
The text was updated successfully, but these errors were encountered:
The internal value for IntegerInput, DecimalInput, and other derived inputs, is stored as a JavaScript number. In most (all?) implementations, numbers are stored as double-precision binary floating point values. However, integer values are stored Python-side as Python ints (or longs), which have exact integer precision, and infinite range (although Axiom has range limits. Also, decimal values are stored as Decimal values on the Python side, which have prec=28 in the default context. Thus, values lose precision when sent from the server to the client; and values entered by the user also lose precision compared with the server-side capability.
The text was updated successfully, but these errors were encountered: