Can I train closed-loop time-series network in parallel?

I have implemented a NARX recurrent neural network.

I have implemented it first as an open loop network, and then train it using the parallel computing capability. This works as expected.

However, when I try to use the parallel computing capability to train my net as a closed-loop, my computer hangs indefinitely.

Answer 1

[MathWorks Support Team]

Note that parallelism happens across samples, or in the case of time series across different series. However, if the network has only input delays, with no layer delays, the delayed inputs can be precalculated so that for the purposes of computation, the time steps become different samples and can be parallelized. This is the case for networks such as "timedelaynet" and open-loop versions of "narxnet" and "narnet".

If a network has layer delays, then time cannot be 'flattened' for purposes of computation, and so single series data cannot be parallelized. This is the case for networks such as "layrecnet" and closed-loop versions of "narxnet" and "narnet". However, if the data consists of multiple sequences, it can be parallelized across the separate sequences.

See this documentation page for more information:

https://www.mathworks.com/help/deeplearning/ug/neural-networks-with-parallel-and-gpu-computing.html#btldb55-1