how to remove data discontinuities, jumps, outliers, and other artifacts that are not noise. Smoothing is bad,
23 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Eugene Davis
el 22 de Jul. de 2016
Comentada: Walter Roberson
el 24 de Jul. de 2016
I'd like to hear of cool ways to process what should nominally be smooth data and detect and remove jumps, single point outliers, and other artifacts that are not noise. The data must be knit together prior to doing a nonlinear regression fit to a model.
0 comentarios
Respuesta aceptada
Walter Roberson
el 23 de Jul. de 2016
You do not know what is noise and what is not until you have a model to compare against. You therefore cannot eliminate the noise in order to fit to determine what your model is.
With a non-linear model, a value that is much different than the others is not necessarily an artifact or noise. For example it might turn out that the model was y = 1/(a*x-b)^2 in which case if x turned out to be very close to b/a then the y value could be quite big but just a very short distance away y might be much smaller. Eliminating the large y value would, in such a case, lead to an incorrect model.
2 comentarios
Walter Roberson
el 24 de Jul. de 2016
You might know the parametric form, but we do not know it. In the general case with non-linear systems, you cannot know what the noise is until you have produced a complete model with coefficients.
You want to do "outlier detection" to find points that cannot be produced by any combination of parameters for the form of model that you have. That requires a bunch of numeric analysis.
Perhaps the following might help:
https://www.mathworks.com/matlabcentral/fileexchange/34795-outlier-detection-and-removal--hampel-
https://www.mathworks.com/matlabcentral/fileexchange/37211-remove-outliers
https://www.mathworks.com/matlabcentral/fileexchange/28501-tests-to-identify-outliers-in-data-series http://www.mathworks.com/help/signal/ref/hampel.html
Más respuestas (1)
Image Analyst
el 23 de Jul. de 2016
Let's say you have a step and you remove it. What will be in it's place? It can't be a step anymore because you removed it. So it will be smoother. It must be because there is no longer a step there. How can you remove a step without smoothing the data? Please show an illustration of that.
Ver también
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!