How to import data with different structures

Hello,

I've a whole bunch of datafiles existing of text and tables. These files are filled automatically with measurement data and because of this the size is different for each file.

I want to import the data to analyze and plot the content of the different tables.

At the moment I use textscan and load the file in one cell...

[name, pathname] = uigetfile('*.log');
filename = cat(2, pathname, name);
delimiter = '\t';
formatSpec = '%s%[^\n\r]';
fileID = fopen(filename,'r');
dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'MultipleDelimsAsOne', true, 'EmptyValue' ,NaN, 'ReturnOnError', false);
fclose(fileID);
Data = dataArray{:, 1};
clearvars delimiter formatSpec fileID dataArray ans;

Then find the beginning and end of each table with strcmp

Q = strcmp('qqqqqq', Data);
x = strcmp('xxxxxx', Data);

Then reload the data again with textscan with begin and end row defined as Q and X and load the content of the table into double and cells (depends on the content). This is repeated for all tables .

My question: I'm sure there's a easier and less time-consuming process but I don't know how. Could someone explain to me what's the best way to handle this kind of logfiles

best regards, loen

Answer 1

[Ronit]

Hello,

To streamline and optimize the workflow for parsing and analyzing the log files with variable-length table, please refer to the following approach:

1. Read the entire file as "Text" using the "fileread" function.

[name, pathname] = uigetfile('*.log');
filename = fullfile(pathname, name);
fileText = fileread(filename);
lines = strsplit(fileText, '\n');

2. Use the markers to find the start and end of each table.

startIdx = find(contains(lines, 'qqqqqq'));
endIdx = find(contains(lines, 'xxxxxx'));

3. Extract each table block and parse it based on its content.

tables = {};
for i = 1:length(startIdx)
    tableLines = lines(startIdx(i)+1 : endIdx(i)-1);
    % Convert to numeric or cell depending on content
    if contains(tableLines{1}, ',') || contains(tableLines{1}, '\t')
        data = cellfun(@(x) strsplit(x, '\t'), tableLines, 'UniformOutput', false);
    else
        data = tableLines;
    end
    tables{end+1} = data;
end

4. Use "cell2table" function to convert each parsed block to MATLAB table.

Refer to the following documentation pages of "fileread" and "cell2table" functions for more details:

• https://www.mathworks.com/help/matlab/ref/fileread.html
• https://www.mathworks.com/help/matlab/ref/cell2table.html

Thanks,

Ronit