Inconsistent forecasts using LSTM deep learning model

Hi everyone

I have a LSTM model setup. The model makes future predictions for 5 days. I ran the model multiple times on the same input data. Each time the output is different. I am new to AI so I am learning as I go. Is the forecast supposed to change if the inputs have not changed?

Thank you

Answer 1

[Prasanna]

Hi Manny,

Different predictions from a LSTM model on the same input data across many runs is due to a variety of factors as mentioned below:

• Neural networks typically start with randomly initialized weights. Different initializations can lead to different local minima during training, resulting in different predictions.
• If you're using stochastic gradient descent (SGD) or any of its variants (like Adam), the training process involves randomness (e.g., random shuffling of data, mini-batch selection).
• Some operations in deep learning libraries can be non-deterministic depending on the environment which leads to varied results.

Essentially, Machine learning algorithms in general are non-deterministic. This has to do with the random initialization of the weights. If you want to make the results reproducible you have to eliminate the randomness from the table.

To achieve consistent results, you can set a random seed to ensure that the random initialization and other stochastic processes are consistent across runs. You can use the ‘rng’ function in MATLAB to set random seed for reproducibility. However, note that even with a fixed seed, some non-deterministic operations might still cause slight variations. Also, once you have trained a model that gives satisfactory results, save the model weights and load them for future predictions to ensure consistency.

For more information regarding the above, refer to the following documentations:

• ‘rng’: https://www.mathworks.com/help/matlab/ref/rng.html
• how to save and reuse a trained neural network: https://www.mathworks.com/matlabcentral/answers/264160-how-to-save-and-reuse-a-trained-neural-network
• How to save any trained machine learning model to use it for prediction later?: https://www.mathworks.com/matlabcentral/answers/511740-how-to-save-any-trained-machine-learning-model-to-use-it-for-prediction-later

[Manny]

Hi @Prasanna

Thank you for this. It is very helpful.

I have a question. In your reply you mention randomness a number of times. My script is getting data from SQL Server. One thing about SQL Server is that data that is stored in a table doesn't always follow the order that was used to create the table. For example, suppose there is tableA with 3 fields - name, date, phone number. I sort by name and then by date and then insert this into tableB. When I retrieve the results from tableB the order of name followed by date will not be respected. The records will not be in any particular order. I have to re-sort tableB. In my LSTM script, I am explictly asking MATLAB to sort the records.

    CIVSymbol = sortrows(CIVSymbol,"QUOTE_DATE","ascend");

This is removing the randomness of the data. Should I be doing this in MATLAB? Or should I just use the data as is from SQL Server without the above command?

Thank you