Strange result from comparison

1 view (last 30 days)
Sachi
Sachi on 17 Nov 2022
Edited: Stephen23 on 17 Nov 2022
Sorry for the naive question. Surely there is a trivial answer. Why does this happen? Thanks.
>> 12*10^-3==0.012
ans =
logical
1
>> 12*10^-4==0.0012
ans =
logical
0
  1 Comment
Sachi
Sachi on 17 Nov 2022
>> 12*10^-5==0.00012
ans =
logical
0
>> 12*10^-6==0.000012
ans =
logical
1

Sign in to comment.

Accepted Answer

Stephen23
Stephen23 on 17 Nov 2022
Edited: Stephen23 on 17 Nov 2022
"Why does this happen?"
Because of the accumulated floating point error. Lets have a look at the values involved, to a higher precision:
fprintf('%.64f\n',12*10^-3,0.012) % these look the same...
0.0120000000000000002498001805406602215953171253204345703125000000 0.0120000000000000002498001805406602215953171253204345703125000000
fprintf('%.64f\n',12*10^-4,0.0012) % but are these the same?
0.0012000000000000001117161918529063768801279366016387939453125000 0.0011999999999999998948757573558054900786373764276504516601562500
Always remember that:
  • binary floating point arithmetic has some similarities to the algebra and arithmetic that you learned at school, but it is definitely not the same thing.
  • the values you see displayed by default in the command window are to a relatively low precision, in general they do not show numbers with their full precision (see NUM2STREXACT).
Learn more about binary floating point numbers:
This is worth reading as well:

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!