Saving simulation data during training process of RL agents

21 views (last 30 days)
Hello everyone
Using the Logging tab of the scope blocks inside the Simulink, I managed to save the data to the MATLAB workspace and then save them to a text file. I need to do the same thing for every iteration (episode) during an RL agent's training process. I wonder how I can similarly save the log data for further analysis and visualization.

Answers (2)

Ari Biswas
Ari Biswas on 19 May 2021
Elaborating on Emmanouil's suggestion:
There are two ways to log and visualize data during training.
Option 1 is to use the training statistics structure (output of the train function). You must enable the Signal Logging checkbox under Model Settings > Data Import/Export. Then specify the signals to log in the Simulink model. This can be easily done by right clicking a signal and selecting Log Selected Signals. After the training is complete you can access this data from the SimulationInfo field in the training statistics struct.
Option 2 is to use the Simulation Data Inspector (SDI). With this option you can visualize the data as the simulations are completed during training. You would need to enable the Record logged workspace data in SDI option under Model Settings > Data Import/Export in the Simulink model, then specify all signals to log in the model. If training in parallel, execute the following command prior to starting training for making the SDI compatible with parallel simulations. Parallel workers will send logged data to the SDI as the simulations are completed.
Simulink.sdi.enablePCTSupport('local') % if the parallel workers are from the local cluster
The SDI also has options to export logged data to the MATLAB workspace.

Emmanouil Tzorakoleftherakis
Have you tried logging the data with Simulation Data Inspector? Make sure to pick only signals you actually need since depending on duration of training, you might end up saving a lot of data.
  2 Comments
Nima Mathwork
Nima Mathwork on 1 Feb 2021
Thank you @Emmanouil Tzorakoleftherakis for your answer. I think using Simulation Data Inspector is a proper solution. However, I could not find an appropriate example in the documentation. Now, I managed to use the cell array, which is suggested here. I would be grateful if you can indicate an example for using Simulation Data Inspector. As a reminder, I need the raw data to draw my custom plots.
Emmanouil Tzorakoleftherakis
Maybe this page and link included are helpful. I don't think there is an example specifically for reinforcement learning in the documentation.

Sign in to comment.

Categories

Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange

Products


Release

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!