AethSmoothing

Smoothing function for aethalometers

AethSmoothing smooths aethalometer data based on Pikridas et al., 2019, which in turn was based on Hagler, G.: Post-processing Method to Reduce Noise while Preserving High Time Resolution in Aethalometer Real-time Black Carbon Data, Aerosol Air Qual Res, doi:10.4209/aaqr.2011.05.0055, 2011.
The algorithm uses uncorrected data (no Weigartner or any other correction) as delivered by the instrument or calculated by the user. Only homogeneous time series are considered.The original code by Hagler et al. can be found in the supplementary info of the paper cited above. It is written in matlab as well.

Syntax:
[response_smoothed,response_std,positions, NumberOfPoints ] = AethSmoothingUpload(Time,Reponse,Attenuation);

Inputs:
Time, Mx1 vector. Time of the data in julian format, ie 1 = one day
Response, Mx1 matrix The response to be smoothed, it can be babs or BC mass in any units
Attenuation, same size as Response. The attenuation as provided by the instrument or calculated by the user (Important read note below). This input must have enough accuracy for the function to operate properly. 4 decimal accuracy is optimal. 3 decimal accuracy will suffice in most cases.

VARARGIN:

'method', the method to use for smoothing. Possible options are
{'moving'}
'lowess'
'sgolay'
'average'
The moving method is better tested. The lowess method is very problematic. The average method will smooth similar to the original ONA algorithm based on the reference above (ie by Hagler et al., 2011).

'ATNincrement', A number below unity. Arranges the intensity of the smoothing. The
greater the value the more intense the smoothing. A default value
of 0.05 is used if the increment is not set.

'Weights', {0} or 1. This parameter is only valid if the moving method is used. In case the sample has large variations in the concentration of BC the method may yield unrealistic results. Applying weights eliminates such problems. However, applying weights may increase the effect of outliers. So it is advised for both the normal and weighed method to be used in parallel. Applying weights will
distort the edges (beginning and end) of your time series.

Output:
response_smoothed, The response smoothed based on ATNincrement
response_std, one standard deviation of the smoothed response positions, the positions of the data smoothed. This output helps duplicate the result. However, when you use weights you cannot duplicate the result

NumberOfPoints, how many data points were used for each average.

% Example:
[smooth_moving,smooth_moving_std,smooth_moving_pos,smooth_moving_np ] = AethSmoothing(Time,babs,Attennuation);
% the above statement is equals to
[smooth_moving,smooth_moving_std,smooth_moving_pos,smooth_moving_np ]...
= AethSmoothing(Time,babs,Att,'method','moving','ATNincrement',0.05,'Weights',0);

[smooth_moving,smooth_moving_std,smooth_moving_pos,smooth_moving_np ] = AethSmoothing(Time,babs,Attennuation, 'Weights',1);
will do the same but now weights are applied.

% [smooth_moving,smooth_moving_std,smooth_moving_pos,smooth_moving_np ] = AethSmoothing(Time,babs,Attennuation, 'ATNincrement',0.01);
reduces the ATN increment to 0.01 from 0.05, which is the default.

******************** *****************************Read Me - Very important notes ********************************************************************
- This function will not work properly if the attenuation does not have enough important digits. So if you ask your time series to be smoothed with ATNincrement equal to 0.01 this means the attenuation input must have 3 decimal digits accuracy. This is very important for AE51 aethalometer model, which requires ATNincrement equal to 0.01 but the output attenuation is 2 digit accuracy and the does not work. In this case, the user MUST calculate attenuation based on measurement and reference signals also provided in the output. Check Pikridas et al., 2019 AMT to learn how to do that.

- Time series must be homogeneous. By homogeneous it is meant time series with an even step throughout. As an example, STAP output is not homogeneous because even though it is evenly spaced, it contains gaps every 3-4 measurements when 1sec resolution is sought. The user must fill the gaps, eg by linear interpolation.

- When you apply smoothing the aim is to produce a realistic output.
However, when the sampling air mass concentration changes rapidly, smoothing may alter the result so that it no longers resembles the actual conditions.