Discontinuity in plot after applying moving average filter

Hello, I am somewhat a beginner in matlab and I am having trouble obtaining a continuous plot after applying a moving average filter to the dataset;

The following graph is the original data with no filter:

Whereas the following graph is the filtered data using the smoothdata function in matlab:

As you can see, a discontinuity appears in the upper right corner of the graph, is there a way to solve this?

Note that the data consists of 100 points

The following is the code used to obtain the graphs:

% Load the mat file
data = load('TADataset.mat');

% Access the cells in the struct
cellArray = data.TADataset;

% Preallocate variables to store the fields
numCells = numel(cellArray);
AnnulusAutoPoints = cell(numCells, 1);
AnnulusManualPoints = cell(numCells, 1);
AnnulusManualPointsmm = cell(numCells, 1);
fileNames = cell(numCells, 1);

% Loop through the cells and extract the fields
for i = 1:numCells
    % Access the struct in the cell
    structData = cellArray{i};
    
    % Extract the fields
    AnnulusAutoPoints{i} = structData.AnnulusAutoPoints;
    AnnulusManualPoints{i} = structData.AnnulusManualPoints;
    AnnulusManualPointsmm{i} = structData.AnnulusManualPointsmm;
    fileNames{i} = structData.UniqueFilename;
end

% Access specific fields for a particular struct
% Example: Accessing the fields for the first struct
firstAnnulusAutoPoints = AnnulusAutoPoints{1};
firstAnnulusManualPoints = AnnulusManualPoints{1};
firstAnnulusManualPointsmm = AnnulusManualPointsmm{1};
firstFileName = fileNames{1};

% Set the window size for the moving average
windowSize = 5;

% Apply manually implemented moving average filter
smoothedFirstAnnulusAutoPoints = smoothdata(firstAnnulusAutoPoints, 'movmean' , windowSize);

% Plot the original data
figure;
plot3(firstAnnulusAutoPoints(:, 1), firstAnnulusAutoPoints(:, 2), firstAnnulusAutoPoints(:, 3), 'r-');
xlabel('X');
ylabel('Y');
zlabel('Z');
title('Original AnnulusAutoPoints');
grid on;
view(-37.5, 30);

% Plot the smoothed data (manual implementation)
figure;
plot3(smoothedFirstAnnulusAutoPoints(:, 1), smoothedFirstAnnulusAutoPoints(:, 2), smoothedFirstAnnulusAutoPoints(:, 3), 'b-');
xlabel('X');
ylabel('Y');
zlabel('Z');
title('Smoothed AnnulusAutoPoints');
grid on;
view(-37.5, 30);

% Plot both on the same figure
figure;
hold on;
plot3(firstAnnulusAutoPoints(:, 1), firstAnnulusAutoPoints(:, 2), firstAnnulusAutoPoints(:, 3), 'r-');
plot3(smoothedFirstAnnulusAutoPoints(:, 1), smoothedFirstAnnulusAutoPoints(:, 2), smoothedFirstAnnulusAutoPoints(:, 3), 'b-');
xlabel('X');
ylabel('Y');
zlabel('Z');
title('Original & Smoothed AnnulusAutoPoints');
grid on;
view(-37.5, 30);
legend('Original data', 'Smoothed data ', 'Location', 'Best');

[Sai Teja G]

Hey, can you add the TADataset.mat file, so that we can run the code and check the error.

[Shadi Al Aridi]

This is the TADataset.mat file:

Answer 1

[Nathan Hardenberg]

This discontinuity is to be expected! That is what the moving average does. The ends gets averaged also, and thus do get shifted away from the original datapoints. Look at the small example below.

One way to circumvent this is to pad the array before and after with the first/last value. That is what I have done below. Note that the graph is shifted because the datavector now gets longer and it is only one dimensional

data = [1 2 4 5 4 2 1]

data = 1×7

1 2 4 5 4 2 1

data_movAvg = movmean(data, 3)  % note the not matching endpoints in the result

data_movAvg = 1×7

1.5000 2.3333 3.6667 4.3333 3.6667 2.3333 1.5000

% Pad the array
padding = ones(1,2);
% -----------|..pad before...|.data|...pad after......|
dataPadded = [padding*data(1), data, padding*data(end)]

dataPadded = 1×11

1 1 1 2 4 5 4 2 1 1 1

dataPadded_movAvg = movmean(dataPadded, 3) % apply moving average on the padded data

dataPadded_movAvg = 1×11

1.0000 1.0000 1.3333 2.3333 3.6667 4.3333 3.6667 2.3333 1.3333 1.0000 1.0000

figure; clf; hold on;
plot(data);
plot(data_movAvg);
plot(dataPadded_movAvg);
legend(["original data","moving average","padded moving average"])

If you know that your data is continuous you can look at the answer linked below. There the moving average gets applied as if the data is a ring. This is the way to do a moving average on a "circle" and it is better that just padding the array.

https://de.mathworks.com/matlabcentral/answers/511963-moving-average-considering-the-first-and-end-data-points-without-changing-window-size-cyclic-window

Still this would most likely give you a visual discontinuity. But now it is only something visual. If you do not like it you can simply append the first datapoint as the last one, and your plot will be connected

Answer 2

[Sai Teja G]

Hi Shadi,

I understand that you are getting discontinuous graph when you used smoothdata.

The discontinuity is due to the values of the smoothdata, and there is no error in the graph. By examining the values of 'smoothedFirstAnnulusAutoPoints', you can observe the same pattern in the plot.

The discontinuity occurs because the end values, which are the moving means of the last windows, have closer values. I have attached an image for reference, and you can also check the values in the command window. If you want a continuous graph, just change the last coorinates of smoothed data to start coordinates of the same.

Hope it helps!