How can I smooth this plot?

So i have a large data set. I just copied here one set, as an example. I tried to use filters to get rid of those small spikes, but i ended up with just a completely straight line.

I split this data in thre sectoins (720 length each) , because I want to kee that stepping characteristic of it.

I am using R2017b edition of matlab, so there are many functions i cannot use to do this.

Could anyone help me with this?

Answer 1

[Star Strider]

Try this:

[D,S] = xlsread('Book1.xlsx');
signal = D(:,1);

L = size(signal,1);
Fs = 1;                                                             % Sampling Frequency (Default, Time Vector Or Sampling Frequency Not Provided)
Fn = Fs/2;                                                          % Nyquist Frequency
msig = signal - mean(signal);
FTsig = fft(msig)/L;                                                % Subtract Mean (Makes FFT Easier To Interpret)
Fv = linspace(0, 1, fix(L/2)+1)*Fn;                                 % Frequency Vector
Iv = 1:numel(Fv);                                                   % Index Vector

figure
plot(Fv, abs(FTsig(Iv))*2)
grid
xlim([0  0.05])

% Fs = 2250;                                                          % Sampling Frequency
% Fn = Fs/2;                                                          % Nyquist Frequency
Wp = 0.019*Fs/Fn;                                                   % Stopband Frequency (Normalised)
Ws = 1.2*Wp;                                                        % Passband Frequency (Normalised)
Rp =  1;                                                            % Passband Ripple
Rs = 60;                                                            % Passband Ripple (Attenuation)
[n,Wp] = ellipord(Wp,Ws,Rp,Rs);                                     % Elliptic Order Calculation
[z,p,k] = ellip(n,Rp,Rs,Wp,'low');                                  % Elliptic Filter Design: Zero-Pole-Gain 
[sos,g] = zp2sos(z,p,k);                                            % Second-Order Section For Stability

figure
freqz(sos, 2^18, Fs)                                                % Filter Bode Plot
set(subplot(2,1,1), 'XLim',[0 Fs/5])                                % Optional
set(subplot(2,1,2), 'XLim',[0 Fs/5])                                % Optional

signal_filt = filtfilt(sos, g, signal);                             % Filter Signal

t = linspace(0, 1, L)/Fs;                                           % Time Vector

figure
plot(t, signal, '-b')
hold on
plot(t, signal_filt, '-r', 'LineWidth',1.5)
hold off
grid
xlabel('Frequency')
ylabel('Amplitude')
legend('Original Signal', 'Filtered Signal', 'Location','SW')

It designs a lowpass filter that filters out most of the noise:

The stopband ‘Ws’ is set to be a specific multiplier of the passband frequency, so you only need to change ‘Wp’ to get different passband and stopband frequencies.

Experiment to get the result you want.

[Star Strider]

Note: All the functions my code uses existed in R2017b. There should be no problems running it.

[Lala0099]

Thank you so much! It does work!

One question I have, is if I did not have those step like decreases in the graph. Just a simple spiky signal, would i still have to use elliptic filter? Or could simplify this code ?

[Star Strider]

My pleasure!

It depends on the result you want.

I would definitely use the elliptic filter. It is relatively straightforward to implement, and is very effective at eliminating frequency-specific noise, such as in your signal. (It would likely be even more effective without having to deal with the step transitions.) You would need to use the Fourier transform of your signal (in figure(1) in my code) to decide on the most optimal passband frequency, then experiment to get the detail you need in your filtered signal with as little noise as possible.

[Star Strider]

If my Answer helped you solve your problem, please Accept it!

[Lala0099]

I did more background reading on elliptic filters and now it makes sense! Thank you !

[Star Strider]

As always, my pleasure!

Answer 2

[Hank]

A little more crudely than Strider's answer, since you may not have some of those functions in R2017...

Here is a function that cuts a function up at significant discontinuites and smooths the individual parts. If you don't have the smooth function, replace it with a simple one from the file exchange like https://www.mathworks.com/matlabcentral/fileexchange/19998-fast-smoothing-function

function xs = smoothsmallstuff(x,w)
sig = 6;
x = x(:);

dx = diff(x);
mdx = mean(dx);
sdx = std(dx);

idiscont = [1; find(abs(dx-mdx)/sdx>6); length(x)];

xs = x;
for i = 1:length(idiscont)-1
    sect = idiscont(i):idiscont(i+1);
    xs(sect) = smooth(x(sect),w);
end
end

[Lala0099]

Thank You for your help!

I tried the smooth function before but it did not work. If i dont have those step like decreases in my output, just a simple signal with spikes, then how could I smooth the signal ?

[Hank]

Did you include the, smoothing width when you used the function?

smooth(x,width) % width = index-width of moving average. Default is 5 points

The default smoothing width is only 5 points. Since your data is really large >1000 points, 5pt smoothing appear effective. Try:

smooth(x,300)

Note this will also smooth out larg discontinuities, hence the function i wrote above.