MATLAB Answers

Reinforcement learning deployment in real-time system

39 views (last 30 days)
Hello everyone,
I trained an agent on my problem, and it works really well in simulation in Simulink. I'm trying to implement the same agent in the real-time target; for that reason, I need to build my Simulink file to generate code and then upload it to my hardware. Unfortunately, I got an error in building my model. I was wondering if anyone can help me with this.
### Generating code into build folder: G:\My Drive\00Research\RL\QUBE_DC_control\Deployment\DC_motor_RL_v02_exp_grt_rtw
Dot indexing is not supported for variables of this type. Function 'evaluatePolicy.m' (#38.286.341), line 13, column 11: "coder.loadDeepLearningNetwork('agentData.mat','policy')" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment. Function 'evaluatePolicy.m' (#38.365.371), line 15, column 19: "policy" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed. Function 'evaluatePolicy.m' (#38.140.167), line 7, column 11: "localEvaluate(observation1)" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed. Function 'MATLAB Function1' (#24.45.62), line 3, column 16: "evaluatePolicy(u)" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'DC_motor_RL_v02_exp/MATLAB Function1'
Component:MATLAB Function | Category:Coder error
### Build procedure for model: 'DC_motor_RL_v02_exp' aborted due to an error.
Errors occurred during parsing of MATLAB function 'DC_motor_RL_v02_exp/MATLAB Function1'

  0 Comments

Sign in to comment.

Accepted Answer

Emmanouil Tzorakoleftherakis
Edited: Emmanouil Tzorakoleftherakis on 21 Sep 2020
Hello,
To generate code from a trained policy, you should follow the process shown here. Note that this is a MATLAB-based workflow, i.e. you cannot generate code directly from Simulink prior to 20b, unless you import the network into your Simulink model through an S-function. Here is a video and an example that works through the workflow with GPU Coder.
In 20b, you can bring a neural network in Simulink with the deep learning block library (see video) and generate code directly from Simulink (see example).

  2 Comments

Armin Norouzi
Armin Norouzi on 20 Sep 2020
Thank you for your response. I created the code using the posted link. This resulted in "evaluatePolicy_mex.mexw64", but I am not sure how I can use it. I tried the legacy code tool to create S-function, but unfortunately, there isn't any good example/documentation to do that.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!