MappedTensor disk-mapped variables and lazy loading

A class that represents data remaining on disk with a very low memory foot-print, but behaves as a normal numeric array.

21 descargas

Actualizado 17 Aug 2021

Ver licencia

This is a class that creates a "numeric variable" representing some data remaining on disk. This way you can transparently treat data larger than the available memory, aka lazy loading and memory-mapped disk data.
You can:
  • create an array of ANY size
  • populate objects with huge data (stored on disk)
  • use many math operators (131 methods)
  • import from: EDF, HDF5, Analyze, ADSC Detector, MAR CCD, MAT 7.3, MRC CCP4, NifTI, NumPy NPY NRRD, Uncompressed TIFF and tomography VOL/PAR.
  • export to: HDF5, Analyze, NumPy NPY, NRRD and NifTI.
m1 = load(MappedTensor, 'hugefile1.nrrd');
m2 = MappedTensor('hugefile2', [1000 1000 1000],'Format','double');
m3 = m1+m2; % sizes must match
save(m3, 'hugefile_sum.hdf5:/Data'); % creates a 'Data' entry
This work was initiated by D. Muir at and is now highly improved (operators, load, save). It benefited from the HDF5 mem mapped object, and more generally is inspired by the memmapfile Matlab object, but in a much more advanced way.
Warning : of course, operators will be slower than when done in memory. Expect an order of magnitude slower.

Citar como

Emmanuel Farhi (2023). MappedTensor disk-mapped variables and lazy loading (, MATLAB Central File Exchange. Recuperado .

Compatibilidad con la versión de MATLAB
Se creó con R2012b
Compatible con cualquier versión desde R2012b
Compatibilidad con las plataformas
Windows macOS Linux

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!



Versión Publicado Notas de la versión

Allow to manipulate an array of object (for instance a stack of TIFF images) as a meta-array of higher dimensionality. Few bug fixes.


Fix in Data setting (introduced with HDF5/MAT support).


Support for disk-mapped/lazy loading of MAT files.


Better handling of HDF5 files : Can now use '?query' in a URL-style to search for attributes.