cross product is wrong
15 views (last 30 days)
Rik on 5 Aug 2021
Edited: Rik on 5 Aug 2021
This number is so small that you can assume this is a rounding error.
The root cause of this rounding error is that it is not possible to store infinite decimals in finite computer memory.
Imagine you can only store decimal numbers, and only 4 digits after the decimal point.
(1/3)*3 = (0.3333)*3 = 0.9999
Is * wrong? Or /? Yes and no. You just need to be aware of the limitations of the tools you're using.
Jan on 6 Aug 2021
Geogebra treats the dta as symbolical expressions. Matlab converts -2159/277 to a numericalvalues as default. If you want to use symbolic calculations, use sym().
Do not check numerical floating point values to be exactly 0 in a condition of an if command. Calculations with numerical values include rounding effects, which cannot be neglected:
1e17 + 1 - 1e17 % 0
1e17 - 1e17 + 1 % 1
sin(2 * pi) % -2.4493e-16
There are no well defined fixed limits and it depends on the application what you have to "consider as 0".
A numerical algorithm requires an analysis of the rounding effects to estimate the reliability of the result.