Numbers, units, and bits
Why can 0.1 + 0.2 differ from 0.3?
Tools to use
The total should be 0.3, yet the display says 0.30000000000000004. Rounding to two decimal places looks better, but does not necessarily solve the underlying problem. Representation and the stage at which rounding occurs can affect totals and comparisons. Investigate with small expressions before merely hiding the extra digits.
Use the decimal calculator for arithmetic on entered decimal values and the floating-point inspector to examine their binary32 or binary64 encoding. They answer different questions even when given the same value. Compare them to choose a representation suitable for the task rather than declaring one universally correct.
Start with the intended decimal expression
0.1 + 0.2The result is 0.3. Enter 0.1 + 0.2 - 0.3 using ASCII operators and obtain 0. This calculator treats decimal literals as fractions and avoids ordinary Number arithmetic for intermediate operations. Non-terminating results still require output precision and rounding; high precision does not mean infinitely many displayed digits.
Compare your program using the same input strings. A number copied after another operation rounded it is already a different input. Record incoming data, the evaluated expression, and final display rounding separately to locate the first divergence.
What happens when 0.1 must fit into binary digits?
In the floating-point inspector, choose decimal input and binary32, then enter 0.1. The stored value displays as 0.10000000149011612 with hexadecimal 3dcccccd. The finite representation selects a nearby value rather than exactly the entered decimal 0.1. This is a representation constraint, not a typing error.
Now enter 0.5: the same format stores 0.5 with hexadecimal 3f000000. Not every fraction incurs this approximation. The important distinction is that a finite decimal expansion need not have a finite binary expansion. Nor should you assume every operation has the same error merely because 0.1 is problematic.
A display of 0.1 does not prove exact storage
Switching to binary64 displays 0.1 and hexadecimal 3fb999999999999a. The shorter displayed number does not prove that exact decimal 0.1 is stored internally. The inspector exposes both a human-readable value and stored bits precisely because these are different views.
This distinction matters in comparisons: two values displayed as 0.3 may not be internally equal. Adding the same tiny constant everywhere is not a general fix. If using tolerances, derive them from the quantity's scale and required accuracy rather than changing monetary or counting rules to accommodate floating-point behavior.
Precision means significant digits here
The calculator's precision setting is a significant-digit count. With precision 3 and half-up rounding, 12345 becomes 12300 and 0.012345 becomes 0.0123. It does not mean three places after the decimal point. Keep that distinction in mind before using the output as a fixed-decimal monetary display.
12345
0.012345
1 / 3
(1 / 3) * 31 / 3 displays as 0.333, but (1 / 3) * 3 produces 1 because the expression retains intermediate fractions until output. Copying the displayed 0.333 and multiplying by 3 instead gives 0.999. Passing only visible intermediate digits into another calculation changes the input and therefore the calculation.
Rounding modes diverge at a tie
At precision 2, compare half-up and half-even for 1.25: they yield 1.3 and 1.2. Half-even resolves an exact tie toward an even retained digit. For 1.35 both yield 1.4, so agreement on one example does not establish identical rounding rules.
For negatives, 'round down' can be ambiguous. At precision 2, -1.29 becomes -1.2 toward zero but -1.3 toward negative infinity. Distinguish reducing magnitude from moving left on the number line. If a specification says only 'truncate', establish whether negatives occur and obtain explicit expected values.
Round each item or round the total?
Consider adding two values of 1.25. Rounding each to two significant digits with half-up gives 1.3 + 1.3 = 2.6. Adding first and rounding the total gives 2.5. Matching the rounding mode is insufficient when its timing differs. For taxes, fees, or rewards, also establish whether rounding occurs per item or per order.
The calculator cannot decide the correct order; reproduce the contract or calculation specification. Keep original inputs because rounded values cannot restore discarded fractions. Minimizing intermediate rounding is not universally correct either: omitting required per-item rounding changes the prescribed calculation.
Report the expression and rounding conditions
Record input strings, expression, number format, significant digits, rounding mode, and stage. '1.25 at precision 2 with half-even gives 1.2' is reproducible; 'decimals look wrong' is not. Shortening displayed digits does not change intermediate types or the precision of transferred data.
Test a simple sum, a non-terminating division, an exact tie, a negative value, and rounding before versus after summation. These distinguish representation issues from business rounding rules. The calculator checks an expression; it does not establish measurement accuracy or compliance with a tax or pricing scheme.
Exact arithmetic cannot improve the original measurement
If a sensor displays only tenths, exact arithmetic can faithfully add those entered strings but cannot recover what the sensor rounded away. More output digits do not provide more knowledge about the measured object. A ten-digit answer does not create ten-digit measurement accuracy. This calculator does not propagate measurement uncertainty, so establish the input's origin separately.
An amount of 0.10 fixed by a contract is different from an approximate sensor reading. Identical-looking decimals may be exact contractual values, measurements or rounded results from another screen. Avoiding floating point alone does not resolve these meanings. Consider storage, intermediate arithmetic and final display together with the origin of the input when defining precision and rounding rules.