# precision of double variables

40 views (last 30 days)

Show older comments

The code below outputs 1 and 1.1 (and not 0.9):

for ii = -1.1:0.1:1.1

if ii >= 0.9

disp(ii)

end

end

If the first line is replaced as the following:

for ii = -1.2:0.1:1.2

the output becomes 0.9, 1, 1.1 and 1.2.

The difference betwwen the actual value of the variable ii and the expected value is epsilon=2.2204e-16.

Does anyone have any idea why replacing 1.1 by 1.2 causes the behavior?

##### 0 Comments

### Accepted Answer

Walter Roberson
on 18 Jun 2022

Suppose you were using decimal to 4 decimal places.

1/3 == 0.3333

1/3 + 1/3 = 0.3333 + 0.3333 = 0.6666

2/3 = 0.6667 because that is the closest representable number to 4 decimal places

Notice that taking the closest representation to 1/3 and adding it to itself does not get you the closest representation to 2/3

You could clearly expand this to any finite number of decimal places. 1/3 is 3 repeated any finite number of times, add it to itself to get 6 repeated that number of times, but 2/3 is 6 repeating ending in 7.

This is thus an inherent problem in using any fixed numeric base to a finite number of places: there will exist some number that rounds up when calculated as a fraction but rounds down through repeated addition.

It happens that MATLAB uses industry standard IEEE 754 binary floating point, which is what is designed into your computer CPU (if you have a GPU then the GPU might be nearly the same but differ on handling very small magnitude numbers near 1e-308)

IEEE 754 binary floating point (and every other binary floating point) has the property that 1/10 cannot be exactly represented. This is for the same mathematical reasons that decimal cannot exactly represent 1/3 or 1/7 or 1/11: every system that represents numbers with finite precision in a fixed numeric base has the same deficiency for some numbers.

So 0.1+0.1 in binary floating point is not guaranteed to be exactly the same as starting with 0.2 directly.

The next thing you need to know is that "for" loops operate by cumulative addition. First value plus increment. Add the increment again. Add the increment again. And so on.

"for" loops do not work by taking the starting point and multiply the increment by the number of iterations and add to the base: that has slightly different effects on accumulated loss of precision.

##### 0 Comments

### More Answers (2)

David Goodmanson
on 17 Jun 2022

Edited: David Goodmanson
on 17 Jun 2022

Hi Arda,

you pretty much answered the question by mentioning precision. Most floating point numbers are not represented exactly in memory. That includes most rational numbers such as .9. So you can't always expect the floating point values in the for loop to exactly agree with a value established in some other way. In the following example, you would at least expect the two would-be values of .9 to agree with each other:

% save the values of ii using concatenation (not recommended but the list is small here)

iivals = [];

for ii = -1.1:0.1:1.1

iivals = [iivals ii];

end

iivals(21) - .9 % 21st value should be .9

ans = -1.1102e-16

a = (-1.1:0.1:1.1);

a(21) - .9

ans = 1.1102e-16

but they don't, and neither equals Matlab's best approximation to .9. The for loop value of ii is too small, so .9 doesn't get displayed. (Doing the 1.2 case will show that the value of ii is just over the line so .9 does get displayed).

To see how these values are stored in memory (Matlab uses the IEEE754 standard) you can use format hex:

format hex

iivals(21)

a(21)

.9

ans = 3feccccccccccccc % too small

ans = 3fecccccccccccce % too large

ans = 3feccccccccccccd

The fix is pretty simple: don't use floating point values in an indexing situation. For example, something like

for ii = -11:11

x = ii/10 % use this value to calculate stuff

if ii >= 9 % use ii for equality-type checks

end

avoids a lot of problems.

##### 20 Comments

Walter Roberson
on 24 Jun 2022

Jan
on 17 Jun 2022

Edited: Jan
on 17 Jun 2022

Welcome to the world of numerics with limited precision.

These are the expected effects. You observe the value mentioned in the frequently asked question:

If you start at -1.1, you see the deviation at 1.0:

for ii = -1.1:0.1:1.1

if ii >= 1.0

disp(ii)

end

end

If you start at -1.2 you see it at 0.9, so this is simply shifted by 0.1 .

##### 0 Comments

### See Also

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!