Reinforcement learning deployment in real-time system

7 views (last 30 days)
Hello everyone,
I trained an agent on my problem, and it works really well in simulation in Simulink. I'm trying to implement the same agent in the real-time target; for that reason, I need to build my Simulink file to generate code and then upload it to my hardware. Unfortunately, I got an error in building my model. I was wondering if anyone can help me with this.
### Generating code into build folder: G:\My Drive\00Research\RL\QUBE_DC_control\Deployment\DC_motor_RL_v02_exp_grt_rtw
Dot indexing is not supported for variables of this type. Function 'evaluatePolicy.m' (#38.286.341), line 13, column 11: "coder.loadDeepLearningNetwork('agentData.mat','policy')" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment. Function 'evaluatePolicy.m' (#38.365.371), line 15, column 19: "policy" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed. Function 'evaluatePolicy.m' (#38.140.167), line 7, column 11: "localEvaluate(observation1)" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed. Function 'MATLAB Function1' (#24.45.62), line 3, column 16: "evaluatePolicy(u)" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'DC_motor_RL_v02_exp/MATLAB Function1'
Component:MATLAB Function | Category:Coder error
### Build procedure for model: 'DC_motor_RL_v02_exp' aborted due to an error.
Errors occurred during parsing of MATLAB function 'DC_motor_RL_v02_exp/MATLAB Function1'

Accepted Answer

Emmanouil Tzorakoleftherakis
Edited: Emmanouil Tzorakoleftherakis on 21 Sep 2020
Hello,
To generate code from a trained policy, you should follow the process shown here. Note that this is a MATLAB-based workflow, i.e. you cannot generate code directly from Simulink prior to 20b, unless you import the network into your Simulink model through an S-function. Here is a video and an example that works through the workflow with GPU Coder.
In 20b, you can bring a neural network in Simulink with the deep learning block library (see video) and generate code directly from Simulink (see example).
  3 Comments
Justus Nwoke
Justus Nwoke on 4 Aug 2022
I have a similar problem: https://www.mathworks.com/matlabcentral/answers/1774210-deploying-trained-agent-to-run-in-simulink-hardware-in-the-loop-self-balancing-robot

Sign in to comment.

More Answers (0)

Categories

Find more on Multicore Processor Targets in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!