MATLAB Answers

Importing large .csv files into MATLAb but confined by limits of excel

40 views (last 30 days)
So I am using an Arduino to collect many values over a long period of time. I am storing these values in a CSV file. Excel does not let you exceed 1048576 rows in one file. However, that is only 8 minutes worth of data for me and I want to collect about 24 hours of data. Now I thought, I will just open the file straight in matlab, surely matlab doesnt have a restriction such as this one. To my dismay, when I open up my file in MATLAB it only shows me 1048576 rows, just like excel. So does matlab have a restriction? When I open the disk the .csv file is written on, there is a small excel icon next to it (by default I am supposing). Does this mean when I import it into Matlab, it is importing it not only as a .csv file but also as an excel file, and the restrictions of excel are imposed upon it? I am using uigetfile to open it up. Thank you

  1 Comment

Stephen Cobeldick
Stephen Cobeldick on 25 Jul 2017
uigetfile does not load any data files, so it is unclear how you are loading this data into MATLAB.

Sign in to comment.

Accepted Answer

Titus Edelhofer
Titus Edelhofer on 25 Jul 2017
Hi Patrick,
the importer you mention will use xlsread in the background and yes, would be limited by the same limit.
But: nobody stops you from reading the file directly, the easiest will be to use a datastore:
doc datastore
Hope this helps,
Titus

  4 Comments

Show 1 older comment
Titus Edelhofer
Titus Edelhofer on 25 Jul 2017
Exactly. xlsread uses Excel as ActiveX Server application. It's convenient to use to import csv files, therefore using Excels engine. But it's not needed, datastore (or even textscan as a somewhat lower level function) reads csv files as text files, not through Excel.
Walter Roberson
Walter Roberson on 25 Jul 2017
If you use the 'basic' option of xlsread then it will not use excel. Also readtable will not use excel
Katrina Anderson
Katrina Anderson on 7 Jan 2019
Hi,
I am having the same issue. However, I am uploading data from an IMU device and I think it is uploaded on to the computer as a .csv
Is there anyway I can get around that?
Thanks
Katrina

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by