How can I predict output values higher than those on the training set using LSTM netoworks?

Dear all,

I am trying to forecast time series using Matlab. I am obtaining good results using LSTM networks for sequence forecasting. Future output values are well predicted until they grow over those in the training set. At that point, output looks truncated to the highest value the network has known during training.

Is there a setting I should look into to let the output grow out of this boundary?

[Santosh Fatale]

HI José María,

Could you share sample code which you used and associted dataset to understand the issue you are encountering?

[Santosh Fatale]

Hi Jose Maria,

What is the last layer in LSTM network?

P.S. : Please post your answers to comments into Comment section and not in Answers section.

[José María Cámara Nebreda]

The structure of the LSTM network is as follows:

layers = [ ...

sequenceInputLayer(featureDimension)

lstmLayer(numHiddenUnits,'OutputMode','sequence')

fullyConnectedLayer(10)

lstmLayer(numHiddenUnits2,'OutputMode','sequence')

dropoutLayer(0.0)

fullyConnectedLayer(numResponses)

regressionLayer];

where numHiddenUnits = 50

numHiddenUnits2 = 10

numresponses = 1

Answer 1

[José María Cámara Nebreda]

Hi Santosh,

Many thanks for your interest.

Unfortunately we are unable to share our dataset at this point in our research.

We are using a LSTM network to predict future behaviour of our output variable based on a previous training over a certain period of time. The predictions are accurate no matter how far we look into the future but, when the real values of the output grow larger or smaller than those observed during the training period, the predicted magnitud appears to stick to the highest values previously seen. The phenomenon can be observed in this image:

Real values are in blue and predicted values in brown. Prediction always matches reality in shape but upper and lower values are always within the previously learned.

We have tried many combinations of network settings: lstm layers, number of hidden units, dorpout layer, etc, but, even thogh these changes affect training times, accuracy and the like, this phenomenon appears unchanged.

I understand that this may not be enough information for you to advise and I apollogize for it. Any hint you could provide would be appreciated though.