matlab digit precision is not correct?
Mostrar comentarios más antiguos
Quoting from:
By default, MATLAB® uses 16 digits of precision.
But why when I write 900000000+2 (8 zeros after 9) it returns 900000002 but writing 900000000+2 (9 zeros after 9)returns 9.0000e+09 isn't this an 8 digit precision?
Respuestas (1)
James Tursa
el 7 de Abr. de 2020
Editada: James Tursa
el 7 de Abr. de 2020
That is just a display artifact. The entire number is still there in memory. Do this to change the display format
format longg
and then repeat your test.
1 comentario
Ahmad Agbaria
el 8 de Abr. de 2020
Categorías
Más información sobre Logical en Centro de ayuda y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!