How to run a program on an csv file data in batches

3 views (last 30 days)
Hello
I am trying to import data from csv file with size (1046520 1). Using the importdata function.
K = load('caamel.csv');
It is taking over an hour to compute my program.
How do I run my code on the excel file in batches instead of one whole data which is taking forever to see result
I will appreciate your help
thanks
Tino
  12 Comments
Walter Roberson
Walter Roberson on 20 Jun 2019
You implied that the reason that your code is taking so long is that import (but really load) is slow. I showed timing on a file 50 times larger than yours could take about 20 minutes to load() so your file would take about half a minute to load(). If you were to switch to textscan then it should take roughly 250/50 = 5 seconds. At that point you might as well read in all the data and loop over portions of it.
textscan also permits you to to specify a count to read, if you find that easier than loading everything and looping indexing.

Sign in to comment.

Answers (0)

Tags

Products


Release

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by