MATLAB Answers

Use of textscan instead of load in a for loop

3 views (last 30 days)
Hi,
I'm at the moment loading data using the following code:
data = cell(1,n_sim);
for m_a = 1:n_sim
for m_b = 1:n_seed
data{1,m_a}{m_b} = load(fileList{1,m_a}{m_b}, 'A1:end');
end
end
This takes a long time since there are a lot of files and they are quite big. The resulting data cell is a 1xn_sim cell, and each cell has 1xn_seed cells inside. Each of those i.e. data{1,1} contains a 11000x25 double.
I would like to use textscan to get the same format on my results. The files that I'm reading have 25 columns and 11000 rows. Can I get some help with the scripting of texscan for this case?
Thanks
  6 Comments
Walter Roberson
Walter Roberson on 7 Sep 2018
Is the data space delimited or comma delimited ?

Sign in to comment.

Accepted Answer

Walter Roberson
Walter Roberson on 7 Sep 2018
ncol = 25;
fmt = repmat('%f', 1, ncol);
data = cell(1, n_sim);
for m_a = 1:n_sim
data_mb = cell(1, n_seed);
for m_b = 1:n_seed
fid = fopen('fileList{1,m_a}{m_b}', 'rt');
data_mb(m_b) = textscan(fid, fmt, 'CollectOutput', 1); %notice () not {} on output
fclose(fid);
end
data{1,m_a} = data_mb;
end

More Answers (1)

Alexander Jensen
Alexander Jensen on 6 Sep 2018
I imagine that the reason why it takes a while is due to the lack of preallocation of memory, I would however do like this (I know it is not using textscan, but why would you? :D )
dims = [11000 25];
data = nan(n_sim, n_seed, dims(1), dims(2));
for m_a = 1:n_sim
for m_b = 1:n_seed
data(m_a,m_b,:,:) = load(fileList{1,m_a}{m_b}, 'A1:end');
end
end
  1 Comment
Isma_gp
Isma_gp on 7 Sep 2018
Hi, thanks for this. It is still taking the same amount of time though.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by