Why does rem(a, b) function give different answers for input values' class 'double' and 'uint64'?
1 view (last 30 days)
Show older comments
Omkar Chavan on 28 Oct 2020
Edited: Omkar Chavan on 29 Oct 2020
I define two values as double,
a1 = 9419588158802421600; b1 = 44;
I am trying to use built-in remainder function,
v1 = rem (a1,b1)
output>> v1 = 28
Now, I define the same values but as uint64
a2 = uint64(9419588158802421600); b2 = uint64(44);
and use remainder function
v2 = rem (a2,b2)
output>> v2 = (uint64) 0
Why two different answers are obtained for the same input values?
Steven Lord on 28 Oct 2020
Let's look at the value stored in a1 converted to a uint64.
>> a1 = 9419588158802421600; b1 = 44;
>> a2 = uint64(9419588158802421600); b2 = uint64(44);
>> [uint64(a1); a2; intmax('uint64')]
3×1 uint64 column vector
Not all integer values on the magnitude of 9419588158802421600 can be exactly represented by a double. For example, the next representable number after a1 is:
>> uint64(a1 + eps(a1))
and the one immediately before it:
>> uint64(a1 - eps(a1)/2)
You can see this by looking at the hex representation of the numbers:
>> [a1 - eps(a1)/2; a1; a1 + eps(a1)]
They differ only in the last hex digit. So the computations in double and in uint64 are operating on different data. When you try to compute on numbers that are close to or greater than flintmax be very careful.
More Answers (0)
Find more on Logical in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!