In a NARX, are the weights kept constant from one training process to another with different input data?

Hello everybody,

In my project, I have to develop a NARX to predict a time series based on the information obtained from frames of videos. For each video, my input data is a row cell, where each location corresponds to a frame, i.e. one timestep.

I have many videos, but I can only give to the network one video at a time. What i'm doing is like: give one video, train the narx, give the following video, train the narx, etc. Note that all videos have different number of frames.

So, my question is the following: am I proceeding well, since the weights and all the network information are kept from one training process to another? Or am I losing my time, since the weights are always initialized?

Thank you for helping me :)

Answer 1

[Greg Heath]

You cannot train in batches of different data. The weight updates of the last batch will supercede all previous updates.

Therefore you have to include representative old data with the new data.

Typically, the old data endpoints are more important than the old data central points.

Hope this helps.

Thank you for formally accepting my answer

Greg

[André Brás]

Thank you for your answer, it's clear now.

Answer 2

[Sarah Mohamed]

Assuming that a function such as "adapt" is being used to incrementally train the network, it may still possible for the neural network to "forget" earlier trained inputs even when the weights are passed from one training iteration to the next. You might find the following discussion helpful:

https://www.mathworks.com/matlabcentral/answers/218805-switching-an-existing-batch-trained-nar-network-to-continuous-incremental-training-with-adapt

Padding the shorter sequences and combining the data may be an alternative approach. From the following document,

https://www.mathworks.com/help/nnet/ug/multiple-sequences-with-dynamic-neural-networks.html

it is noted that "...it is required that each sequence be of the same length. If this is not the case, then the shorter sequence inputs and targets should be padded with NaNs, in order to make all sequences the same length. The targets that are assigned values of NaN will be ignored during the calculation of network performance."

[André Brás]

Thank you. As you said, combining the data may be the best solution.