Perform Incremental Learning Using IncrementalClassificationKernel Fit and Predict Blocks

Since R2024b

This example shows how to use the IncrementalClassificationKernel Fit and IncrementalClassificationKernel Predict blocks for incremental learning and classification in Simulink®. The IncrementalClassificationKernel Fit block fits a chunk of observations (predictor data) using a configured incremental kernel model for binary classification (incrementalClassificationKernel) and outputs the updated incremental learning model parameters as a bus signal. The IncrementalClassificationKernel Predict block accepts an incrementalClassificationKernel model and a chunk of predictor data, and outputs the predicted labels for the observations.

Load and Preprocess Data

Load the human activity data set. Randomly shuffle the data.

load humanactivity
n = numel(actid);
rng(0,"twister") % For reproducibility
idx = randsample(n,n);
X = feat(idx,:);
Y = actid(idx);

For details on the data set, enter Description at the command line.

Responses can be one of five classes: Sitting, Standing, Walking, Running, or Dancing. Dichotomize the response by identifying whether the subject is moving (actid > 2).

Y = Y > 2;

Create Incremental Learning Model

Create an incremental kernel model for binary classification. Specify that the data has 60 predictors and that the data type of the responses is logical. Specify to standardize the data using an estimation period of 500 observations. Create a workspace variable kernelMdl to store the initial incremental learning model.

kernelMdl = incrementalClassificationKernel(NumPredictors=60,ClassNames=[false,true], ...
    Standardize=true,EstimationPeriod=500);

Create Input Data for Simulink

Simulate streaming data by dividing the training data into chunks of 50 observations. For each chunk, select a single observation as a test set to import into the IncrementalClassificationKernel Predict block.

numObsPerChunk = 50;
nchunk = floor(n/numObsPerChunk);
numPredictors = size(feat,2);
Xin = zeros(numObsPerChunk,numPredictors,nchunk);
Yin = zeros(numObsPerChunk,nchunk);
Xtest = zeros(1,numPredictors,nchunk);
for j = 1:nchunk
    ibegin = min(n,numObsPerChunk*(j-1) + 1);
    iend = min(n,numObsPerChunk*j);
    idx = ibegin:iend;   
    Xin(:,:,j) = X(idx,:);
    Yin(:,j) = Y(idx);
    Xtest(1,:,j) = X(idx(1),:);
 end

Convert the training and test set chunks into time series objects.

t = 0:size(Xin,3)-1;
Xtrain_ts = timeseries(Xin,t,InterpretSingleRowDataAs3D=true);
Ytrain_ts = timeseries(Yin',t,InterpretSingleRowDataAs3D=true);
Xtest_ts = timeseries(Xtest,t,InterpretSingleRowDataAs3D=true);

Open Provided Simulink Model

This example provides the Simulink model slexIncClassificationKernelPredictExample.slx, which includes the IncrementalClassificationKernel Fit and IncrementalClassificationKernel Predict blocks. The Simulink model is configured to use kernelMdl as the initial model for incremental learning and classification.

Open the Simulink model slexIncClassificationKernelPredictExample.slx.

slName = "slexIncClassificationKernelPredictExample";
open_system(slName);

Simulate Model

Simulate the Simulink model to perform incremental learning and predict responses to the test set observations. Export the simulation outputs to the workspace. You can use the Simulation Data Inspector (Simulink) to view the logged data of an Outport block.

simOut = sim(slName,"StopTime",num2str(numel(t)-1));

% Extract labels
yfit_sig = simOut.yout.getElement(1);
yfit_sl = squeeze(yfit_sig.Values.Data);

% Extract scores values
scores_sig = simOut.yout.getElement(2);
scores_sl = squeeze(scores_sig.Values.Data);

% Extract beta values
beta_sig = simOut.yout.getElement(3);
beta_sl = squeeze(beta_sig.Values.Data);

% Extract bias values
bias_sig = simOut.yout.getElement(4);
bias_sl = squeeze(bias_sig.Values.Data);

At each iteration, the IncrementalClassificationKernel Fit block trains the model and updates the model parameters. The IncrementalClassificationKernel Predict block calculates the predicted label for each test set observation.

Analyze Model During Training

To see how the model parameters and response values evolve during training, plot them on separate tiles.

figure
tiledlayout(4,1);
nexttile
plot(scores_sl(1,:),".")
ylabel("Score")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(yfit_sl,".")
ylabel("Label")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(beta_sl(1,:),".-")
ylabel("\beta_1")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(bias_sl,".-")
ylabel("Bias")
xlabel("Iteration")
xlim([0 nchunk])

During the estimation period, the IncrementalClassificationKernel Fit block estimates hyperparameters but does not fit the initial model (see incrementalClassificationKernel). Therefore, the score output of the IncrementalClassificationKernel Predict block, the model beta coefficients, and the model bias all equal zero during the first 10 iterations. At the end of the estimation period, the IncrementalClassificationKernel Fit block updates the model parameters and predicts labels. After the estimation period, the score values vary between approximately –12 and 23. The first beta coefficient varies significantly during the first 30 iterations following the estimation period, and approaches a constant value of approximately -0.03 thereafter. The bias (intercept) term has an initial value of 0 and gradually approaches 0.3.