Create cell array of files from subfolder

I have a folder with 2 subfolders.

Each subfolder has a variable amount of text files, all containging unique data, in the same dimensions while having the same name.

E.g. Subfolder one = Temp.txt (500x14 table), Pressure.txt (400x10 table), Humidity (450x11 table)

Subfolder two = Temp.txt (500x14 table), Humidity (450x11 table)

I was using a fileDatastore to store these. How can I create a mulitlevel data store like this: fds.Files = 3x1 cell array, cell1 = Temp, cell2 = Pressure, cell3 = Humidity. Each cell being expandable, holding the tables containging the data? So the Temp array cell would be a 2x1 array containing a 500x14 table in cell1 and a 500x14 table in cell2?

I had written a code doing this and foolishly overwritten it this morning and now can't get it back. If there are any problems to this method please let me know. My code is below:

Dir = "C:\...";
Folders = dir(fullfile(Dir));
Folders = Folders(~ismember({Folders.name},{'.','..'}));

Folders = struct2cell(Folders);
Folders = transpose(Folders);
Folders = Folders(:,2);
Folders = unique(Folders);

fds = fileDatastore(Folders, ReadFcn=@(x) readtable(x,VariableNamingRule="preserve"), FileExtensions=".txt", IncludeSubfolders=true, ReadMode="file");
data = readall(fds);

[~,fileName] = fileparts(fds.Files);
Path_Name_Data = cat(2,Folder,fileName,data);

Answer 1

[Pratyush]

Hi @Cathal,

I understand that you want to store tables in your txt file into a mat file in such a way that data of the files having same name under different subfolders of the target folder should be clubed together. Following code should help you:

% Define the root folder path
rootFolder = 'path/to/root/folder';
% Initialize an empty cell array to hold the file structs
fileStructs = {};
% Traverse through the root folder and its subfolders
subfolders = dir(rootFolder);
subfolders = subfolders([subfolders.isdir]); % Filter only directories
for i = 1:numel(subfolders)
    subfolder = subfolders(i).name;
    if strcmp(subfolder, '.') || strcmp(subfolder, '..')
        continue; % Skip current and parent directories
    end
    
    % Get the list of txt files in the current subfolder
    txtFiles = dir(fullfile(rootFolder, subfolder, '*.txt'));
    
    for j = 1:numel(txtFiles)
        txtFile = txtFiles(j).name;
        
        % Extract the file name without the extension
        [~, fileName, ~] = fileparts(txtFile);
        
        % Check if the file name already exists in the file structs array
        fileIndex = find(cellfun(@(x) strcmp(x.name, fileName), fileStructs));
        if isempty(fileIndex)
            % Create a new file struct and add the table to its list
            fileStruct.name = fileName;
            fileStruct.tables = {readtable(fullfile(rootFolder, subfolder, txtFile))};
            fileStructs{end+1} = fileStruct;
        else
            % Append the table to the existing file struct's list
            fileStructs{fileIndex}.tables{end+1} = readtable(fullfile(rootFolder, subfolder, txtFile));
        end
    end
end
% Save the file structs cell array as a mat file
save('data.mat', 'fileStructs');

This would store your data in a data.mat file. This would have an array of struct where each struct would have a "name" and "tables" attribute. "table" is an array of tables from the files having same name.

[Cathal]

Hi Pratyush, thanks. This is what I was looking for