RL Agent Action Limits don't work?

2 views (last 30 days)
Matt
Matt on 26 Jul 2021
Hello.
I am trying to create an RL agent. In order to assess the agent's progress, I have my Environment output significant data when the agent's score is over a particular threshold. However, I have noticed that the Actions undertaken by the RL agent are sometimes out of the bounds I have defined for it. A portion of the code is given below with a subset of examples in bold:
ActionInfo = rlNumericSpec([12 1]);
ActionInfo.Name = 'Stats';
%% Note that I have formatted the action limits into columns here for readability. This is not the case in the actual code. %%
ActionInfo.LowerLimit = [0; 1e-4; -pi; 1e-4; pi/12; -5; 0; 1e-4; -pi; 1e-4; pi/12; -5];
ActionInfo.UpperLimit = [1; 0.01; pi; 2.5e-3; 3*pi; 5; 1; 0.01; pi; 2.5e-3; 3*pi; 5];
Note that the action limits repeat - the agent really has two six-variable actions rather than one 12-variable action. I have extracted the 12-item action vector and found some discrepancies with the assigned action limits. I have included it below, with the erroneous portions in bold:
>> Action'
ans =
1×12 single row vector
0.3871 0.3644 -0.1965 -0.0921 5.4891 0.1257 0.3498 -0.9896 0.6873 0.3430 4.3061 -0.1232
I think it might be worth noting that the errors occur in the same indicies in both cases and that in each case, exponential notation is used. In any case, the values are clearly out of the allowed bounds.
What's going on here? The UpperLimit and LowerLimits are pretty straigtforward, so what's causing it to go out of bounds? Any ideas on how I can fix the issue?
Thanks!

Answers (0)

Products


Release

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!