Non-uniform Discrete Data Sample Filtering

Hello,

I have collected some data from my encoder and IMU and I am trying to come up with a calibration function. However the data is quite noisy (especially IMU). I timestamped then numerized the timestamp using datenum function. I would like to have a spline like output.

I have come across this question and trying to utilize the code shared.

D = load('test_Data.mat');
t = D.test_data(:,1);
s = D.test_data(:,2);
Fs = 1;                                                             % Sampling Frequency
Ts = 1;                                                             % Sampling Interval
[sr,tr] = resample(s, t, Fs);                                       % Resample, Return Resampled Signal & New Time Vector
sre = sr(1:end-2);                                                  % Eliminate End Transient
tre = tr(1:end-2);                                                  % Eliminate End Transient
figure
plot(t, s)
hold on
plot(tre, sre, '--')
hold off
grid
legend('Original Signal', 'Resampled Signal')
L = numel(t);                                                       % Signal Length
Fn = Fs/2;                                                          % Nyquist Frequency
sm = sre - mean(sre);                                               % Subtract Mean
FTs = fft(sm)/L;                                                    % Scaled Fourier Transform
Fv = linspace(0, 1, fix(L/2)+1)*Fn;                                 % Frequency Vector
Iv = 1:numel(Fv);                                                   % Index Vector
figure
plot(Fv, abs(FTs(Iv))*2)
grid
title('Fourier Transform')
Wp = [0.05]/Fn;                                                     % Passband Frequency (Normalised)
Ws = [0.09]/Fn;                                                     % Stopband Frequency (Normalised)
Rp =  1;                                                            % Passband Ripple
Rs = 60;                                                            % Passband Ripple (Attenuation)
[n,Wp] = ellipord(Wp,Ws,Rp,Rs);                                     % Elliptic Order Calculation
[z,p,k] = ellip(n,Rp,Rs,Wp,'low');                                  % Elliptic Filter Design: Zero-Pole-Gain 
[sos,g] = zp2sos(z,p,k);                                            % Second-Order Section For Stability
figure
freqz(sos, 2^16, Fs)                                                % Filter Bode Plot
sre_filt = filtfilt(sos, g, sre);                                   % Filter Signal
figure
subplot(2,1,1)
plot(tre, sre)
grid
title('Resampled Signal')
subplot(2,1,2)
plot(tre, sre_filt, '-')
grid
title('Filtered Resampled Signal')

However resampling results singular, a single result; not an array.

[Star Strider]

It is difficult to determine what to do with these data.

You need to determine the most appropriiate resampling rate and how to handle all the NaN values that result from creating a data set with the constant sampling intervals necesssary for any sort of signal processing. See the documentation on the retime function for details on its options.

I leave it to you to choose how best to work with them —

LD = load(websave('calib_data1','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1102160/calib_data1.mat'))

LD = struct with fields:
    enc_time: [77552×1 double]
     enc_val: [77552×1 double]
    imu_time: [4514×1 double]
     imu_val: [4514×1 double]

enc_time = LD.enc_time;
enc_val = LD.enc_val;
DT = datetime(enc_time, 'ConvertFrom','datenum');
DT.Format = 'HH:mm:ss.SSSSSS';

T1 = table(DT,enc_val)

T1 = 77552×2 table
          DT           enc_val
    _______________    _______

    16:32:23.674000     0.65  
    16:32:23.674000     0.65  
    16:32:23.674000     0.65  
    16:32:23.674000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  
    16:32:23.721000     0.65  

TT1 = table2timetable(T1);
Fs = 500;                                                       % Resampling  Frequency
TT1 = retime(TT1,'regular','SampleRate',Fs)

TT1 = 54120×1 timetable
          DT           enc_val
    _______________    _______

    16:32:23.674000     0.65  
    16:32:23.676000      NaN  
    16:32:23.678000      NaN  
    16:32:23.680000      NaN  
    16:32:23.682000      NaN  
    16:32:23.684000      NaN  
    16:32:23.686000      NaN  
    16:32:23.688000      NaN  
    16:32:23.690000      NaN  
    16:32:23.692000      NaN  
    16:32:23.694000      NaN  
    16:32:23.696000      NaN  
    16:32:23.698000      NaN  
    16:32:23.700000      NaN  
    16:32:23.702000      NaN  
    16:32:23.704000      NaN  

figure
plot(DT, enc_val)
grid
title('Original')

figure
plot(TT1.DT,TT1.enc_val, '.', 'MarkerSize',0.1)
grid
title(sprintf('After ''retime'' (Fs = %0.1f Hz)',Fs))

The filtering will be relatively straightforward after the data are prepared for it.

Any NaN values in the input vector will result in the entire filttered output being NaN.

.

[Kerem Asaf Tecirlioglu]

I have handled the NaN value by replacing with the nearest integer sample. I am attaching the arrays .

a=1;
q=1;

for i = 1:size(TT1.imu_val)
if isnan(TT1.imu_val(i))
    a = a + 1;
else
    for x = q:i
        TT1.imu_val(x) = TT1.imu_val(i);
    end
    a = 1;
    q = 1 + x;
end
end

I have never used matlab for filtering before. How should I proceed with filtering

[Mathieu NOE]

hi

for filtering look for example for smoothdata

Answer 1

[Maximilian Schönau]

I would reccomend you using the live script task "Smooth Data". There you can graphically try out different filter methods and after that convert your favorite filter to code.