Building an LSTM Regression Model with Sliding Windows

Hi everyone,

I'm working on a regression problem (not forecasting) using LSTM in MATLAB, and I could really use some guidance. Here’s the setup of my project:

Data Overview:

• I have three datasets stored in CSV files.
• The inputs are in columns 3 and 6.
• The output (target) is in column 8.

Sliding Window Setup:

• I want to group the input data into sliding windows of size 15.
• For example, rows 1–15 will correspond to the output at row 16, rows 2–16 to the output at row 17, and so on.

Train-Test Split:

• I plan to use datasets 1 and 2 for training.
• Dataset 3 will be used for testing.

Objective:

• Train an LSTM model that can predict the target values for the test dataset based on the inputs in columns 3 and 6 with the defined window size.

Questions:

1. How can I efficiently implement the sliding window for inputs and outputs in MATLAB?
2. What is the best way to prepare XTrain, YTrain, XTest, and YTest in a cell array format that MATLAB's LSTM accepts?
3. Any tips on how to structure the LSTM for this kind of regression task?
4. How do I configure the LSTM to ensure it doesn’t use predicted outputs as inputs for subsequent predictions?

Any example code snippets, references, or advice would be highly appreciated!

Thanks in advance for your help!

Answer 1

[Aastha]

Hi @ALEJANDRO,

As I understand you want to format a dataset using sliding windows for LSTM regression model training into train and test datasets and store them in cell arrays.

To format your datasets into “XTrain”, “XTest”, “YTrain” and “YTest” you can create a function that takes as input the dataset and gives the sliding window dataset as an output:

function [X, Y] = createSlidingWindows(data, inputCols, targetCol, windowSize) 
    numSamples = size(data, 1) - windowSize; 
    X = cell(numSamples, 1); 
    Y = cell(numSamples, 1); 
    for i = 1:numSamples 
        X{i} = data(i:i+windowSize-1, inputCols)'; 
        Y{i} = data(i+windowSize, targetCol)'; 
    end 
end 

You can create the “XTrain”, “XTest”, “YTrain” and “YTest” variables by calling the above defined function as follows:

% Create sliding windows 
[XTrain1, YTrain1] = createSlidingWindows(data1, [3, 6], [8], 15); 
[XTrain2, YTrain2] = createSlidingWindows(data2, [3, 6], [8], 15); 
[XTest, YTest] = createSlidingWindows(data3, [3, 6], [8], 15); 
XTrain = [XTrain1; XTrain2]; 
YTrain = [YTrain1; YTrain2]; 

To structure your LSTM model for a regression task, it is important to choose the final layer, and an activation function based on the nature of the output variable.

If you are predicting a probability mass function (PMF), use a softmax activation in the final layer to ensure the outputs sum to 1 and represent valid probabilities.

For strictly positive quantities, a fully connected (FC) layer followed by a ReLU activation is ideal, as it ensures non-negative predictions. When predicting continuous values that can be positive or negative, a simple linear activation is often used. These choices help ensure the model produces appropriate outputs for your specific regression task.

You may refer to this link of MathWorks documentation mentioned below for any further information on the LSTM model layers and training parameters:

https://www.mathworks.com/help/deeplearning/ug/long-short-term-memory-networks.html

For more information on ReLU activation function and softmax activation function, refer to the links mentioned below:

-ReLU :

https://www.mathworks.com/help/deeplearning/ref/dlarray.relu.html

-softmax:

https://www.mathworks.com/help/deeplearning/ref/softmax.html?s_tid=doc_ta

You may refer to the steps mentioned below to configure the LSTM:

1. Initialize the LSTM state by making predictions over the first few steps of the input data. This can be done by selecting an offset value and using the first offset steps of the test data to set the network's state. You may refer to the MATLAB code mentioned below to do so:

offset = 15; 
[Z, state] = predict(net, XTest(1:offset, :)); % net is the trained LSTM network 
net.State = state; 

2. You can use the LSTM network to forecast the remaining time steps:

num_time_steps = size(XTest, 1); % get the size of the test data 

num_time_steps = num_time_steps - offset; % subtract the offset data 

Y = zeros(num_time_steps, numChannels); % initialize a variable for the predicted output 

Y(1, :) = Z(end, :); 

 

for t = 1:num_time_steps - 1 

    Xt = XTest(offset + t, :); 

    [Y(t + 1, :), state] = predict(net, Xt); % perform an open-loop prediction 

    net.State = state; 

end 

Using this approach, the LSTM predicts each time step using only the input data, with the state updated at each step based on the model’s predictions.

You can refer to the example given below for more information on open loop and closed loop prediction

https://www.mathworks.com/help/deeplearning/ug/time-series-forecasting-using-deep-learning.html

Hope this is helpful to you!