readtimetable from a java stream instead of file

1 visualización (últimos 30 días)
Marc Vaillant
Marc Vaillant el 18 de Feb. de 2020
Comentada: Marc Vaillant el 24 de Abr. de 2020
Why the heck doesn't matlab provide overloads for IO functions, e.g. readtimetable, that take a low level stream? In particular, is there a way to have readtimetable read from a lowel level file stream instead of a file?
Timetables and tables are nice datastructures but there is a significant roadblock to their interoperability with other languages. Their binary serialized form in .mat files is proprietary, so no one knows how to read/write them from other languages. You can read/write them as ascii--using say csv as a data exchange format--via readtimetable/writetimetable, but then their size blows up by a factor of 3 or 4. When you have timetables that would be 500 MB as mat files, and all of a sudden they become 2gb, that's a problem. You can compress the csvs, but alas, there's no real way to read compressed files on the fly using standard parsing functions like readtimetable. There's this which consoildates the information out there on trying read zip files on the fly without uncompressing and writing to a file first, which basicaly gets you a java file stream that can read from a zip file directly, but you can't pass it to readtimetable. The only option would be to spin your own csv to timetable parser that operates on a java file stream, but that seems silly given that the functionality is already there in readtimetable. Is there another option?
  2 comentarios
Peter Perkins
Peter Perkins el 14 de Abr. de 2020
Marc, you are correct, and it's something that we are looking at. Have you tried using parquet? I confess that I am not really up on parquet, but my understanding is that lots of software can create that format, and in recent versions (looks like R2019a) there are readparquet and friends.
Marc Vaillant
Marc Vaillant el 24 de Abr. de 2020
Thanks Peter. I haven't tried using parquet, but that seems like a reasonable compact data exchange format. I will take a look.
Thanks again,

Iniciar sesión para comentar.

Respuestas (0)


Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by