Deleting rows in table in .mat variable by time condition

Hi, could anyone please help me with this problem ? I have tried a number of things, but not yet achieved anything that would work. I would need a code that would load multiple .mat files, each file has lblSet variable. ClassName column in each file includes either '''Detection'' or ''Seizure'''. In addition, each files includes, among others, also rows Start and End with time information in format hh:mm:ss. My goal is to detect time ranges of those rows that have ''Seizure'' in ClassName column and delete rows with ''Detection'' in ClassName which have time ranges of Start and End within those of Seizure. Then I would need to save only those filtered files that include ''Detection'' in ClassName.

Thanks in advance if there will be someone who could help. I have already spent too much time on this :-)

Enclosed below please find the sample of data and my failed coding attempt.

My attempt - does not work correctly:

function deleteRowsByCondition
    % Get file paths for .mat files
    [filePaths, ~, ~] = getFilepnAllCell('Select label files', 'mat');
    
    % Get the folder path to save the results
    savePath = uigetdir('Select the folder to save the results');
    
    % Preallocate results vector
    iedResults = NaN(size(filePaths));

    % Loop through each file
    for kf = 1:numel(filePaths)
        % Try to load the file and catch any errors
        try
            load(filePaths{kf}, 'lblSet');
        catch ME
            % Display the error message and continue to the next file
            disp(['Error loading file: ' filePaths{kf}]);
            disp(ME.message);
            continue;
        end
        
        % Check if lblSet exists in the loaded file and is not empty
        if exist('lblSet', 'var') && ~isempty(lblSet)
            % Set datetime format for Start and End columns
            lblSet.Start.Format = 'yyyy-MM-dd HH:mm:ss';
            lblSet.End.Format = 'yyyy-MM-dd HH:mm:ss';
            
            % Get the rows with 'Seizure' in ClassName column
            seizureRows = lblSet(strcmp(lblSet.ClassName, "Seizure"), :);
            
            % Get the rows with 'IED_Janca01' in ClassName column
            iedRows = lblSet(strcmp(lblSet.ClassName, "Detection"), :);
            
            % Skip the file if it has 'Seizure' in ClassName column
            if isempty(seizureRows)
                continue;
            end
            
            % Create a matrix with Start and End times and their corresponding values
            timesAndValues = [datetime(lblSet.Start), datetime(lblSet.End); ones(size(lblSet, 1), 1), -ones(size(lblSet, 1), 1)];
            
            % Sort the matrix based on the times
            sortedTimesAndValues = sortrows(timesAndValues', 1)';
            
            % Calculate the cumulative sum of values
            cumSumValues = cumsum(sortedTimesAndValues(3, :));
            
            % Find the indices of the beginnings and ends of seizures
            seizureBeginIndices = find(cumSumValues == 1);
            seizureEndIndices = find(cumSumValues == 0);
            
            % Create a new lblSet for seizures
            lblSetSz = table(sortedTimesAndValues(1, seizureBeginIndices)',...
                             sortedTimesAndValues(2, seizureEndIndices)',...
                             'VariableNames', {'Start', 'End'});
            % Save the modified lblSetSz to a new file
            [~, name, ext] = fileparts(filePaths{kf});
            save(fullfile(savePath, [name '_SzOnly' ext]), 'lblSet');
            
            % Perform the deletion of IED rows during seizures
            deleteRow = false(size(iedRows, 1), 1);
            for j = 1:size(iedRows, 1)
                startTime = iedRows.Start(j);
                endTime = iedRows.End(j);
                % Check if the IED row is within the time range of any seizure row
                if any((startTime >= lblSetSz.Start & startTime <= lblSetSz.End) | ...
                       (endTime >= lblSetSz.Start & endTime <= lblSetSz.End))
                    % Mark the IED row for deletion
                    deleteRow(j) = true;
                end
            end
            
            % Delete the marked IED rows from lblSet
            lblSet(deleteRow, :) = [];
            
        else
            % Display a warning if lblSet is not found or is empty in the file
            warning(['lblSet not found or is empty in file: ' filePaths{kf}]);
        end
    end
end

Answer 1

[Shivam]

Hi Milan,

From the information provided, I understand that you are processing a set of .mat files that have labeled time range data, and you want to delete rows labeled as 'Detection' in the ClassName column that have Start and End times falling within the time range of the rows labeled as 'Seizure.' After filtering out the 'Detection' rows, you want to save the files with the 'Detection' label in the ClassName column.

You can follow the below workaround to make changes in your existing code to achieve the goal:

• Correct loading of lblSet

% Load the lblSet variable properly from the .mat file
data = load(filePaths{kf}, 'lblSet');
lblSet = data.lblSet;

• Proper Datetime conversion

% Convert the Start and End times to datetime objects
lblSet.Start = datetime(lblSet.Start, 'InputFormat', 'HH:mm:ss');
lblSet.End = datetime(lblSet.End, 'InputFormat', 'HH:mm:ss');

• Simplify Overlap Detection

% Initialize a logical array to mark "Detection" rows for deletion
deleteDetection = false(size(detectionRows, 1), 1);

% Loop through each "Detection" row to check for overlaps with "Seizure" time ranges
for i = 1:size(detectionRows, 1)
    for j = 1:size(seizureRows, 1)
        if (detectionRows.Start(i) >= seizureRows.Start(j) && detectionRows.Start(i) <= seizureRows.End(j)) || ...
           (detectionRows.End(i) >= seizureRows.Start(j) && detectionRows.End(i) <= seizureRows.End(j))
            deleteDetection(i) = true;
            break;
        end
    end
end

• Correct deletion of 'Detection' rows

% Remove the marked "Detection" rows from the table
detectionRows(deleteDetection, :) = [];

• Conditional saving of modified files

% Save the modified file with a new suffix if it still contains "Detection" rows
if ~isempty(detectionRows)
    [~, name, ~] = fileparts(filePaths{kf});
    save(fullfile(savePath, [name '_DetectionsFiltered.mat']), 'detectionRows');
end

You can incorporate the above-suggested changes in the code and remove other redundant code involving unnecessary preallocation and cumulative sum.

I hope it helps.

Thanks

[Milan]

Thanks a lot, I will try to rewrite it as you suggested. Looks promising.