Perform Incremental Learning Using IncrementalRegressionKernel Fit and Predict Blocks

Since R2024b

This example shows how to use the IncrementalRegressionKernel Fit and IncrementalRegressionKernel Predict blocks for incremental learning and response prediction in Simulink®. The IncrementalRegressionKernel Fit block fits a chunk of observations (predictor data) using a configured incremental kernel model for regression (incrementalRegressionKernel) and outputs the updated incremental learning model parameters as a bus signal. The incrementalRegressionKernel Predict block accepts an IncrementalRegressionKernel model and a chunk of predictor data, and outputs the predicted response for the observations.

Create Incremental Learning Model

Load the robot arm data set and store the number of observations as n.

load robotarm.mat
n = numel(ytrain);

For details on the data set, enter Description at the command line.

Create an incremental kernel model for regression. Specify that the data has 32 predictors, and standardize the data using an estimation period of 500 observations. Create a workspace variable kernelMdl to store the initial incremental learning model.

rng(0,"twister") % For reproducibility
kernelMdl = incrementalRegressionKernel(NumPredictors=32, ...
    Standardize=true,EstimationPeriod=500);

Create Input Data for Simulink

Simulate streaming data by dividing the training data into chunks of 50 observations. For each chunk, select a single observation from the test set to import into the IncrementalRegressionKernel Predict block.

numObsPerChunk = 50;
nchunk = floor(n/numObsPerChunk);
for j = 1:nchunk
    ibegin = min(n,numObsPerChunk*(j-1) + 1);
    iend = min(n,numObsPerChunk*j);
    idx = ibegin:iend;   
    Xin(:,:,j) = Xtrain(idx,:);
    Yin(:,j) = ytrain(idx);
    Xtestset(1,:,j) = Xtest(j,:);
end

Convert the training and test set chunks into time series objects.

t = 0:size(Xin,3)-1;
Xtrain_ts = timeseries(Xin,t,InterpretSingleRowDataAs3D=true);
Ytrain_ts = timeseries(Yin',t,InterpretSingleRowDataAs3D=true);
Xtest_ts = timeseries(Xtestset,t,InterpretSingleRowDataAs3D=true);

Open Provided Simulink Model

This example provides the Simulink model slexIncRegressionKernelPredictExample.slx, which includes the IncrementalRegressionKernel Fit and IncrementalRegressionKernel Predict blocks. The Simulink model is configured to use kernelMdl as the initial model for incremental learning and prediction.

Open the Simulink model slexIncRegressionKernelPredictExample.slx.

slName = "slexIncRegressionKernelPredictExample";
open_system(slName);

Simulate Model

Simulate the Simulink model to perform incremental learning and predict responses to the test set observations. Export the simulation outputs to the workspace. You can use the Simulation Data Inspector (Simulink) to view the logged data of an Outport block.

simOut = sim(slName,"StopTime",num2str(numel(t)-1));

% Extract responses
yfit_sig = simOut.yout.getElement(1);
yfit_sl = squeeze(yfit_sig.Values.Data);

% Extract CanPredict values
CanPredict_sig = simOut.yout.getElement(2);
CanPredict_sl = squeeze(CanPredict_sig.Values.Data);

% Extract beta values
beta_sig = simOut.yout.getElement(3);
beta_sl = squeeze(beta_sig.Values.Data);

% Extract bias values
bias_sig = simOut.yout.getElement(4);
bias_sl = squeeze(bias_sig.Values.Data);

At each iteration, the IncrementalRegressionKernel Fit block trains the model and updates the model parameters. The IncrementalRegressionKernel Predict block calculates the predicted response for each test set observation.

Analyze Model During Training

To see how the model parameters and response values evolve during training, plot them on separate tiles.

figure
tiledlayout(4,1);
nexttile
plot(CanPredict_sl,".-")
ylabel("CanPredict")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(yfit_sl,".-")
ylabel("Response")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(beta_sl(1,:),".-")
ylabel("\beta_1")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(bias_sl,".-")
ylabel("Bias")
xlabel("Iteration")
xlim([0 nchunk])

During the estimation period, the IncrementalRegressionKernel Fit block estimates hyperparameters but does not fit the initial model (see incrementalRegressionKernel). Therefore, the CanPredict and yfit outputs of the IncrementalRegressionKernel Predict block, the model beta coefficients, and the model bias all equal zero during the first 10 iterations. At the end of the estimation period, the IncrementalRegressionKernel Fit block updates the model parameters and calculates predicted responses. After the first iteration following the estimation period, the response values vary between approximately –0.89 and 1. The first beta coefficient varies significantly during the first 20 iterations following the estimation period, and shows only small variations between –0.001 and 0.001 thereafter. The bias (intercept) term has a high value of 0.02 and gradually approaches zero.