Clear Filters
Clear Filters

Why is there an error in the calculation of 366.0/5.0?

3 views (last 30 days)
I input the following code in Matlab and the result obtained has a small error, as shown below:
>> format long
>> 367/5
ans =
73.400000000000006

Answers (2)

Matt J
Matt J on 1 May 2024
Edited: Matt J on 1 May 2024
All computers make imperfect calculations. It's a fact of life with finite precision floating point math.

Walter Roberson
Walter Roberson on 2 May 2024
By default, MATLAB calculates using IEEE 754 Double Precision representation of numbers. IEEE 754 is a binary floating point representation. One thing about binary floating point representations is that 1/10 is not exactly representable in any finite length binary floating point.
This is not a bug; it is an inherent limitation of binary floating point representation.
The mathematical reasons why 1/10 is not exactly representable in binary is the same reason why 1/3 and 1/6, 1/7, 1/9, and 1/13 are not exactly representable in finite decimal. Every finite base representation has numbers that cannot be exactly finitely represented.
  2 Comments
liao z
liao z on 3 May 2024
I understand that any binary can only perfectly represent 2 ^ (N), and other numerical values can only be represented with finite precision. So what are the significant bits of the double variable?
James Tursa
James Tursa on 4 May 2024
Edited: James Tursa on 4 May 2024
The three closest numbers to 73.4 representable in IEEE double precision are:
format longg
x = [73.4-eps(73.4);73.4;73.4+eps(73.4)];
The binary floating point hex patterns of these three numbers are (they differ by 1 in the trailing bit):
num2hex(x)
ans = 3x16 char array
'4052599999999999' '405259999999999a' '405259999999999b'
The exact decimal conversions of these binary floating point numbers are:
fprintf('%60.50f\n',x)
73.39999999999999147348717087879776954650878906250000 73.40000000000000568434188608080148696899414062500000 73.40000000000001989519660128280520439147949218750000
MATLAB (or more precisely, the CPU floating point processor) picked the closest one for the result. There are no numbers (including 73.4 exactly) inbetween these numbers that are exactly representable in IEEE double precision.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!