Borrar filtros
Borrar filtros

How to specify max. number of outliers on which to apply the Hampel filter?

1 visualización (últimos 30 días)
I have a timeseries with outliers. However, sometimes these occur consequtively, e.g. 20 datapoints one after the other. This poses a problem when applying Hampel on the entire dataset because it alters data that is actually valid.
I would like to apply the Hampel filter only when there are max. 3 outliers in a Hampel window. How can I do that?
The outliers are values within the vector X that exceed a threshold of abs(X) = 10.
Is there a way to create a new vector which keeps datapoints exceeding 10 if most of the sorrounding datapoints are below 10, and if in a window of 20 values, more than 3 of them exceed the threshold, then those extremes should be replaced with 'NaN'?
Thank you in advance!

Respuesta aceptada

Scott MacKenzie
Scott MacKenzie el 5 de Mzo. de 2022
There's probably a simpler way to do this, but I believe the code below achieves what you are after:
% test data
x1 = randi([5 12],1,50);
% filtered data in x2
x2 = x1;
% parameters (adjust as necessary)
ws = 6; % window size
t = 10; % threshold
n = 3; % number of points above threshold (must be >n)
for i=ws:length(x1)
idx1 = i-ws+1;
idx2 = i;
idx = (idx1-1) + find(x1(idx1:idx2) > t );
if length(idx)>n
x2(idx) = nan;
end
end
[x1; x2]
ans = 2×50
7 7 7 7 11 9 10 11 10 5 11 10 12 11 9 12 11 11 8 8 5 11 5 10 7 9 5 6 9 6 7 7 7 7 11 9 10 11 10 5 NaN 10 NaN NaN 9 NaN NaN NaN 8 8 5 11 5 10 7 9 5 6 9 6
  2 comentarios
Lu Da Silva
Lu Da Silva el 5 de Mzo. de 2022
Editada: Lu Da Silva el 5 de Mzo. de 2022
The only correction I made is applying the absolute value:
idx = (idx1-1) + find(abs(x1(idx1:idx2)) > t );
Works wonders, thank you!!

Iniciar sesión para comentar.

Más respuestas (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by