How to visualize episode behaviour with the reinforcement learning toolbox?

How can I create a visualization for a custom environment that shows the behaviour of the system in the environment during an episode of training? I cannot find code examples or clarifations of code that visualizes systems behaviour during training episodes anywhere on Mathworks. I would like to achieve a visualization that looks something like the cart-pole visualizer shown on this page: https://nl.mathworks.com/help/reinforcement-learning/ug/train-pg-agent-to-balance-cart-pole-system.html?searchHighlight=cart%20pole&s_tid=doc_srchtitle.

PS I am trying to solve the continuous mountain car problem with a ddpg agent with the reinforcement learning toolbox

Answer 1

[Emmanouil Tzorakoleftherakis]

Hello,

To create a custom MATLAB environment, use the template that pops up after running

rlCreateEnvTemplate('myenv')

In this template there are two methods that can be used for visualization, "plot", and "envUpdatedCallback" (it is called from within "plot"). Use "plot" to create the basic stationary parts of your visualization, and "envUpdatedCallback" to update the coordinates of the moving parts based on your states.

[Prashanth Chivkula]

I defined the plot function inside the step function. I get multiple figures. It's plotting for each step. If I need to define it outside the step function , can you tell me how to do it outside and how and where to call the function?

[Emmanouil Tzorakoleftherakis]

As I mention above, you should not define or call 'plot' inside step.

Use the "plot" method that exists in that class template to create the basic stationary parts of your visualization, and the "envUpdatedCallback" method to update the coordinates of the moving parts based on your states.

Before training, call

plot(env)

and that's it.

[Prashanth Chivkula]

I get an error stating ''that data should be numeric,datetime, duration or an array convertible to double.''

[Prashanth Chivkula]

And another question where do I define a reward function in the template

[Emmanouil Tzorakoleftherakis]

The error sounds self-explanatory - make sure whatever you are plotting makes sense.

In this template there is no separate function for rewards - it is implemented inside 'step' if you go through the generated code. You could create a separate function if you want as well.

In the future please create a separate question if it's not related to the original one. Thanks!