Save data from subfolders into a cell array

Hi all,

I am trying to access several matrices from my directiry and store them in a cell array.

They are stored in a way where I have 1062 subfolders in my directory that are named with the subject ID. Each of that subfolder contains 4 individual .csv files that are all named the exact same across all subjects.

What I would like to do now is go into each of the subfolder, extract one of the 4 csv.files and save it as a matrix in a cell array. Optimally, the information to which subfolder (aka subject) the matrix belongs would also be included in the cell array.

Any help would be appreciated!

Thanks

Answer 1

[Stephen23]

P = 'absolute or relative path to where the subfolders are';
N = 'the name of the file that you want.CSV';
S = dir(fullfile(P,'*'));
S = S([S.isdir]); % remove files
S(ismember({S.name},{'.','..'})) = []; % remove dot directories
for k = 1:numel(S) % loop over the subfolders
    F = fullfile(P,S(k).name,N);
    S(k).data = readmatrix(F);
end

The imported data are simply stored in the structure S. For example the second file:

S(2).name % subject ID
S(2).data % imported file data

You can trivially loop over the structure and use indexing to process all of the file data.

https://www.mathworks.com/help/matlab/matlab_prog/access-multiple-elements-of-a-nonscalar-struct-array.html

[Johanna Popp]

Thanks a lot for your response!

I tried your approach but I got this error message in the line to remove the dot directories:

Error using cell/ismember (line 34)

Input A of class double and input B of class cell must be cell arrays of character vectors, unless one is a character vector.

[Stephen23]

@Johanna Popp: I used the wrong type of brackets, that line should be:

S(ismember({S.name},{'.','..'})) = [];
%          ^      ^ curly braces

I have corrected my answer. If you are interested how that syntax works, see:

https://www.mathworks.com/help/matlab/matlab_prog/comma-separated-lists.html

https://www.mathworks.com/matlabcentral/answers/320713-how-to-operate-on-comma-separated-lists

[Johanna Popp]

Ah you fixed the brackets, thanks!

The structure is still empty but I think I know where the issue is: I forgot to include in my question that the subject specific folder contains ANOTHER subfolder (named the same exact same for all subjects) that then contains my 4 csv files.. It would be great if you could show a way to include this in the solution, thanks!!

[Stephen23]

"The structure is still empty..."

If you mean that

numel(S)

is zero then no subfolders exist on the path that you gave. Check the path carefully.

"I forgot to include in my question that the subject specific folder contains ANOTHER subfolder (named the same exact same for all subjects) that then contains my 4 csv files.. It would be great if you could show a way to include this in the solution, thanks!!"

You wll need to include the name of the subsubfolder inside the FULLFILE inside the loop, e.g.:

D = 'the name of the subsubfolder';
..
for k = ..
    F = fullfile(P,S(k).name,D,N);
    %                        ^
end

Look at the order I used inside FULLFILE:

• the path to where the subfolders are (constant)
• the subfolder name (subject ID)
• the subsubfolder name (constant)
• the filename (constant)

This is exactly the same as if you were writing the filepath down yourself. Have a look at the output of FULLFIlE (i.e. variable F), it should match what you would see in your explorer address bar or similar tool. You can adjust the folder/file names and order to suit your folder structure.

[Johanna Popp]

Thank you so much for your help!!

[Johanna Popp]

Perfekt, thank you for the explanations!!

Say one of the subjects doesn't have a data entry because the specific csv file that I am looking for is not in the folder. Is the a way to skip subjects?

thanks :)

[Stephen23]

"Is the a way to skip subjects? "

You can test if a particular folder of file exists:

https://www.mathworks.com/help/matlab/ref/isfile.html

and add an IF condition inside the loop when the file data is imported, e.g.:

if isfile(F)
    S(k).data = readmatrix(F);
end

You can do something similar for the subsubfolders too, if required.

[Johanna Popp]

awesome, thanks!