# precision of double variables

40 views (last 30 days)
Arda Yigit on 17 Jun 2022
Commented: Walter Roberson on 24 Jun 2022
The code below outputs 1 and 1.1 (and not 0.9):
for ii = -1.1:0.1:1.1
if ii >= 0.9
disp(ii)
end
end
If the first line is replaced as the following:
for ii = -1.2:0.1:1.2
the output becomes 0.9, 1, 1.1 and 1.2.
The difference betwwen the actual value of the variable ii and the expected value is epsilon=2.2204e-16.
Does anyone have any idea why replacing 1.1 by 1.2 causes the behavior?

Walter Roberson on 18 Jun 2022
Suppose you were using decimal to 4 decimal places.
1/3 == 0.3333
1/3 + 1/3 = 0.3333 + 0.3333 = 0.6666
2/3 = 0.6667 because that is the closest representable number to 4 decimal places
Notice that taking the closest representation to 1/3 and adding it to itself does not get you the closest representation to 2/3
You could clearly expand this to any finite number of decimal places. 1/3 is 3 repeated any finite number of times, add it to itself to get 6 repeated that number of times, but 2/3 is 6 repeating ending in 7.
This is thus an inherent problem in using any fixed numeric base to a finite number of places: there will exist some number that rounds up when calculated as a fraction but rounds down through repeated addition.
It happens that MATLAB uses industry standard IEEE 754 binary floating point, which is what is designed into your computer CPU (if you have a GPU then the GPU might be nearly the same but differ on handling very small magnitude numbers near 1e-308)
IEEE 754 binary floating point (and every other binary floating point) has the property that 1/10 cannot be exactly represented. This is for the same mathematical reasons that decimal cannot exactly represent 1/3 or 1/7 or 1/11: every system that represents numbers with finite precision in a fixed numeric base has the same deficiency for some numbers.
So 0.1+0.1 in binary floating point is not guaranteed to be exactly the same as starting with 0.2 directly.
The next thing you need to know is that "for" loops operate by cumulative addition. First value plus increment. Add the increment again. Add the increment again. And so on.
"for" loops do not work by taking the starting point and multiply the increment by the number of iterations and add to the base: that has slightly different effects on accumulated loss of precision.

David Goodmanson on 17 Jun 2022
Edited: David Goodmanson on 17 Jun 2022
Hi Arda,
you pretty much answered the question by mentioning precision. Most floating point numbers are not represented exactly in memory. That includes most rational numbers such as .9. So you can't always expect the floating point values in the for loop to exactly agree with a value established in some other way. In the following example, you would at least expect the two would-be values of .9 to agree with each other:
% save the values of ii using concatenation (not recommended but the list is small here)
iivals = [];
for ii = -1.1:0.1:1.1
iivals = [iivals ii];
end
iivals(21) - .9 % 21st value should be .9
ans = -1.1102e-16
a = (-1.1:0.1:1.1);
a(21) - .9
ans = 1.1102e-16
but they don't, and neither equals Matlab's best approximation to .9. The for loop value of ii is too small, so .9 doesn't get displayed. (Doing the 1.2 case will show that the value of ii is just over the line so .9 does get displayed).
To see how these values are stored in memory (Matlab uses the IEEE754 standard) you can use format hex:
format hex
iivals(21)
a(21)
.9
ans = 3feccccccccccccc % too small
ans = 3fecccccccccccce % too large
ans = 3feccccccccccccd
The fix is pretty simple: don't use floating point values in an indexing situation. For example, something like
for ii = -11:11
x = ii/10 % use this value to calculate stuff
if ii >= 9 % use ii for equality-type checks
end
avoids a lot of problems.
Walter Roberson on 24 Jun 2022
The third form of for is defined as doing indexing. It would be against the documentation to use any kind of incremental computation for the third form.

Jan on 17 Jun 2022
Edited: Jan on 17 Jun 2022
Welcome to the world of numerics with limited precision.
These are the expected effects. You observe the value mentioned in the frequently asked question:
If you start at -1.1, you see the deviation at 1.0:
for ii = -1.1:0.1:1.1
if ii >= 1.0
disp(ii)
end
end
1 1.1000
If you start at -1.2 you see it at 0.9, so this is simply shifted by 0.1 .

R2021a

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by