datastore for large structured data

There are Datastores for tabular text files, spreadsheet files, image files, etc. What if my data is a very large binary file, with the format: [size1 X double, size2 X double, ...], while the information of "size1, size2..., offset 1, offset 2..." is stored in a different metafile. I can write my own ReadFcn to load the data into a cell array, each cell is a nX1 double array. but it's way too large to fit the memory. I tried to use filedatastore and define my own Readfcn, but I couldn't use it to access portion of the data like the other datastores. Any suggestions?

Answer 1

[Mark Sherstan]

If your file is to big to read into memory consider using the matfile function. You get to write the data straight to the file without loading it into RAM which may help solve your issues.