MATLAB Answers

Reinforcement learning: "NextObs" vs. "LoggedState" in step function

8 views (last 30 days)
Anne Tscheliessnig
Anne Tscheliessnig on 27 Jul 2020
Hi,
I could not find out what the difference between "NextObs" and "LoggedSignals" is in the step function. In all scripts both are passed on from the step function.
[NextObs,Reward,IsDone,LoggedSignals] = myStepFunction(Action,LoggedSignals)
"LoggedSignals" is obviously used for the next step, but what is "NextObs" used for?
Thanks!

  0 Comments

Sign in to comment.

Answers (1)

Emmanouil Tzorakoleftherakis
Actually, NextObs is the important thing here. It represents the value of your states after you apply current action and integrating one step.
LoggedSignals is where you can log information to view later - can be left empty too.

  2 Comments

Anne Tscheliessnig
Anne Tscheliessnig on 28 Jul 2020
The myStepFunction is taking NextObs, Reward, IsDone and LoggedSignal and when calling needs Action and LoggedSignals
function [NextObs,Reward,IsDone,LoggedSignals] = myStepFunction(Action,LoggedSignals)
Then I would need LoggedSignals for the next step? And NextObs is being used for the agent? In this case I could not leave LoggedSignals empty, or?
This example is very confusing for me, because actually it states almost to the end in the myResetFunction
NextObs = LoggedSignals.State;
Emmanouil Tzorakoleftherakis
Oh you were looking at creating custom environments with functions - I was looking at creating environments with classes by running e.g.
rlCreateEnvTemplate('myenv')
where LoggedSignals is not that important since you can use class variables to store the states.
I suspect the reason you need both LoggedSignals and NextObs is to create a unified way of using custom environments regardless of how you create it. NextObs is probably what the agent is using when interacting with the environment, whereas LoggedSignals is a way to save intermmediate values if you don't use classes to create your custom env.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!