I am using the reinforcement learning toolbox to run control tasks, in particular using the DDPG agent. Unfortunately, each episode lasts 100 seconds with a 0.01 s time step (the control time step is 0.1 s, i.e. the RL control block is called that often). The computation time is unfortunately unamangeably high.
I have tried to reduce the training of the actor and critic neural networks to every 5 episodes by using a periodic TargetUpdateMethod and changing the TargetUpdateFrequency. However, by doing a deeper analysis, it is clear that it the computational time taken by each episode, which is too high. So, this is pointing the culpript to the RL Simulink block.
The way I see it, the block should run the neural networks (which is a matrix multiplication) and store the additional experience point in the memory (so some more matrix calculations, if the memory is full). So, this is not fully explaining the large overhead to me.
My code is running (more) efficiently on Python, so it is clear I am not fully exploiting the MATLAB/C++ implementation.
Any advice on how I could try to improve the computational efficiency?