Handling Offsets and Trends in Data
When to Detrend Data
Detrending is removing means, offsets, or linear trends from regularly sampled time-domain input-output data signals. This data processing operation helps you estimate more accurate linear models because linear models cannot capture arbitrary differences between the input and output signal levels. The linear models you estimate from detrended data describe the relationship between the change in input signals and the change in output signals.
For steady-state data, you should remove mean values and linear trends from both input and output signals.
For transient data, you should remove physical-equilibrium offsets measured prior to the excitation input signal.
Remove one linear trend or several piecewise linear trends when the levels drift during the experiment. Signal drift is considered a low-frequency disturbance and can result in unstable models.
You should not detrend data before model estimation when you want:
Linear models that capture offsets essential for describing important system dynamics. For example, when a model contains integration behavior, you could estimate a low-order transfer function (process model) from nondetrended data. For more information, see Process Models.
Nonlinear black-box models, such as nonlinear ARX or Hammerstein-Wiener models. For more information, see Nonlinear Model Identification.
When signals vary around a large signal level, you can improve computational accuracy of nonlinear models by detrending the signal means.
Nonlinear ODE parameters (nonlinear grey-box models). For more information, see Estimate Nonlinear Grey-Box Models.
To simulate or predict the linear model response at the system operating conditions, you
can restore the removed trend to the simulated or predicted model output using the
For more information about handling drifts in the data, see the chapter on preprocessing data in System Identification: Theory for the User, Second Edition, by Lennart Ljung, Prentice Hall PTR, 1999.
Alternatives for Detrending Data in App or at the Command-Line
You can detrend data using the System Identification app and at the command line using
Both the app and the command line let you subtract the mean values and one linear trend from steady-state time-domain signals.
detrend command provides the following additional
functionality (not available in the app):
Subtracting piecewise linear trends at specified breakpoints. A breakpoint is a time value that defines the discontinuities between successive linear trends.
Subtracting arbitrary offsets and linear trends from transient data signals.
Saving trend information to a variable so that you can apply it to multiple data sets.
As an alternative to detrending data beforehand, you can specify the offsets levels as estimation options and use them directly with the estimation command.
For example, suppose your data has an input offset, u0, and an output offset, y0. There are two ways to perform a linear model estimation (say, a transfer function model estimation) using this data:
T=getTrend(data) T.InputOffset = u0; T.OutputOffset = y0; datad = detrend(data, T); model = tfest(datad, np);
Specify offsets as estimation options:
opt = tfestOptions('InputOffset',u0, 'OutputOffset', y0); model = tfest(data, np, opt)
The advantage of this approach is that there is a record of offset levels in the model in
model.Report.OptionsUsed. The limitation of this approach is that it cannot handle linear trends, which can only be removed from the data by using
Next Steps After Detrending
After detrending your data, you might do the following:
Perform other data preprocessing operations. See Ways to Prepare Data for System Identification.
Estimate a linear model. See Linear Model Identification.