Table name from input to loop

Hi,

I'm loading data from different files in a loop. I want to create a table for each individual data file.

datafiles=recursivedir('PATH','*.txt');

for o = 1:length(datafiles)
    %extract date and time from filename
    [versn, name, ext] = fileparts(datafiles{o});
    filename=datafiles{o};

      fileID = fopen(filename,'r');

      dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'MultipleDelimsAsOne', true, 'TextType', 'string',  'ReturnOnError', false);

      fclose(fileID);

      T_'name' = table(dataArray{1:end-1}, 'VariableNames', {'Output','Date','Time'});
% I want to create a new table for each loop with the name set from the fileparts.

      % Clear temporary variables
      clearvars filename delimiter formatSpec fileID dataArray ans;

end

I want to create a table for each loop that has the name like T_test1 and from the fileparts name(1) = 'test1' How do I do this, so I get a table for each loop?

Best regards, Jesper

[Stephen Cobeldick]

Some notes:

• Clearing variables on each loop iteration serves no purpose.
• Using numel rather than length is a good habit to learn.
• It is easy and advisable to check if fopen succeeded.
• textscan returns a cell array of the columns, so why not just use cell2table to get a table?
• Why not just use readtable to import the file directly into a table?
• Dynamically accessing variable names, as you are requesting is not recommended- It is much simpler and more efficient to use a cell array and indexing. Read this to know more:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

[Jesper Kamp Jensen]

Dear Stephen,

Thank you for your answer - there are some helpful things I can use, but the essential part of it for me is to name each table with the filename!! Otherwise, data will be completely compromised and to no use - I have read several of your post on the topic, but it is still essential to me.

Best regards, Jesper

[Stephen Cobeldick]

"but the essential part of it for me is to name each table with the filename!!"

Can you please explain why this is required? So far 99.99% examples I have seen on this and other forums have not required magical naming of variables, even though the OP sometimes claim that theirs is a special case. What is your special case? I am genuinely interested, because I have read a lot of documentation and discussion on this topic, so I am interested to know what you can add to this. Do you have some existing badly-written code that expects lots of separate variables?

In any case, if you really want to write slow, complex, buggy code the main way to achieve this is currently mentioned seventy-four times on my tutorial, something readers of that thread are unlikely to miss.

Note that not all valid filenames are valid variable names, and so your code will be non-robust be design: you could add some processing/checking of these names to make them valid variable names, but of course this just increases the complexity even more. Or you could just avoid the problem entirely by using a cell array.

"Otherwise, data will be completely compromised and to no use"

How? You can trivially store all of the table data and even the tables names using cell arrays and indexing, and trivially access them afterward importing. Absolutely everything that you import is available to you, so how does this "compromise" the data? I regularly import data into cell arrays, and so far I have never observed any of it becoming "compromised" ... by the way, what is "compromised" data?

I look forward to your explanation, and to learning something new!

[Jesper Kamp Jensen]

I run X number of experiments that are stored in txt files and the analyzing equipment names these files according to treatment and sample. There might be an easier way to obtain this, but there is information in the filename that I need to compare different compounds and treatments - if I cannot distinguish between the data files I have no use of the data.

I have used cell arrays and indexing with good results, so I agree with you, but hoped there was a way to incorporate the filenames more easily.

[Stephen Cobeldick]

"but hoped there was a way to incorporate the filenames more easily."

Arrays are the easy way! What you are trying do to is the opposite of easy.

By far the easiest way is to put the filenames into an array, which is what my answer does. Note that meta-data (e.g. filenames) is also data, and so it should definitely not be stored in variable names, because putting (meta-)data into variable names makes it harder to access, and makes your code slow, complex, and more liable to bugs. Read this to know more:

https://www.mathworks.com/matlabcentral/answers/225435-save-variable-as-string-from-user-input

In my answer the filenames are stored in the struct array S, whose contents correspond exactly with the imported data in cell array T, i.e.

S(1).name  <-> T{1}
S(2).name  <-> T{2}
S(3).name  <-> T{3}
... etc

Accessing either/both of these is clearly trivial and simple using indexing. So all of the filenames are there for you, waiting for you to extract whatever information you want from them. You can even use some handy syntaxes for dealing with that non-scalar structure, e.g. to put them all into one cell array:

{S.name}

which you could then use with strcmp, regexp, etc., e.g. to identify filenames starting with 'BMW':

idx = strncmp({S.name},'BMW',3)

Note how simply I can access and process all of the names at once!

If you want all of the data to be collected into one variable then you could even put the tables into the structure:

for k = ...
    ...
    S(k).table = cell2table(...)
end

and then all of the filenames and imported data are in the same structure, and are trivial to access.

[Jesper Kamp Jensen]

Clearly, I do not have enough knowledge nor know-how yet! (But will one ever?)

I thank you very much for your help!! - I completely missed out on what you got the last part! This will do the trick for me - just as you said! :)

Answer 1

[Stephen Cobeldick]

Rather than magically defning variable names it is much simpler to use basic MATLAB indexing and a cell array T:

D = 'Path';
S = dir(fullfile(D,'*.txt'));
T = cell(1,numel(S));
for k = 1:numel(S)
    [fid,msg] = fopen(fullfile(D,S(k).name));
    assert(fid>=3,msg)
    C = textscan(fid,...);
    fclose(fid);
    T{k} = cell2table(C,...);
end