importdata - HELP MEMORY
6 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
I am trying to use importdata to import a .dat file which is approximately 349000KB in size. If I import about 75% of the data, MATLAB keeps his cool... but when I try to import the whole things he gets angry and tells me:
N=importdata('Respiratory_Modulation.dat',' ',1);
??? Error using ==> importdata at 214 Unable to load file. Use TEXTSCAN or FREAD for more complex formats.
Caused by: Error using ==> fileread at 36 Out of memory. Type HELP MEMORY for your options.
Why is he angry with me? What have I done?
Thanks,
Linford
1 comentario
Eric
el 25 de En. de 2012
What happens when you try to create a single variable of this size? For instance, try
A = ones(6700,6700);
(don't forget the semicolon!). If this fails then there really does seem to be a memory issue rather than a problem reading in the DAT file.
Respuesta aceptada
Andreas Goser
el 26 de En. de 2012
Like mentioned in an other answer. A 64 Bit MATLAB on a 64 Bit machine should be tested.
There are however many smart methods to reduce memory. And very often the method depends on your data and application.
In this case, I assume, your data is of type DOUBLE and as you are more than halfway through importing, I like suggesting to convert the data into SINGLE. Most applications do not need DOUBLE data. But as I said, it depends what you do after the data is imported and how often you need to import new data sets.
Assuming, you like to try it:
- Cut the data into half
- import it
- Convert it using SINGLE()
- save it
- Do it with the other half
- Import both
0 comentarios
Más respuestas (2)
Walter Roberson
el 25 de En. de 2012
Not very many systems can store 349 gigabytes of data in RAM. Are you certain you need to load all 349 gigabytes at one time??
0 comentarios
Linford Briant
el 25 de En. de 2012
2 comentarios
Walter Roberson
el 25 de En. de 2012
Just before the importdata() call, for debugging please put in a call
memstats
to see how much memory your system has and how much is available.
(Note: this is only valid for MS Windows.)
Are you using a 32 bit version of MATLAB or a 64 bit version? Would switching to a 64 bit version be practical on your system?
Ver también
Categorías
Más información sobre String Parsing en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!