MATLAB Answers

Reinforcement Learning Toolbox: defining Size of layers and other parameters.

17 views (last 30 days)
Phebe Samuel
Phebe Samuel on 13 Mar 2020
hello,
i am trying to use Reinforcement learning toolbox for a vehicle where i have two inputs /observations for the Reinforcement learning toolbox-DQN Agent and one action to be taken by the DQN Agent. The action has discrete 40 discrete value. Three hiddens layers are to be used with sizes of 20,50 and 100.
Questions:
  1. how to define hidden layers size in the script? is it the size of "fullyConnectedLayer(L,'Name','fc1')"
  2. what should be the size parameter for " imageInputLayer([2 1 1],'Normalization','none','Name','state')" ?
  3. my simulation gives error of
Error using rl.env.AbstractEnv/sim (line 121)
An error occurred while simulating "RL_hev_control"with the agent "agent".
Error in RL_HEVsimulink (line 79)
experience = sim(env,agent,simOptions);
Error using Simulink.SimulationInput/sim Not enough room in the buffer to store the new experiences. Make sure the bufferSize argument is big enough.
What could be the mistake i am doing ?

  1 Comment

Raunak Gupta
Raunak Gupta on 18 Mar 2020
Hi,
Can you share the script (‘.m’ file) and Model (‘.slx’ file) written by you that can reproduce the same issue. It will helpful for me to solve the problem.

Sign in to comment.

Answers (1)

Emmanouil Tzorakoleftherakis
Hello,
I would recomment looking at a similar example in thw documentation (e.g. this one). To your questions:
  1. Yes. Number of neurons are specified as an argument in the fullyConnectedLayer
  2. Reinforcement Learning Toolbox currently uses the imageInputLayer even for non-image inputs (there is ongoing work to add more descriptive layers here). In any case, similar to the example I link to above, your Q network will need one channel for the observations and another for the actions, so you will need to have two imageInputLayers. For the action one, dimension should be [1 1 1], and for the observation you should have [2 1 1].
  3. Is this after training or during training? Looks like it's after training, but at that point the experience buffer should not play a role. This post looks relevant. Have you called 'train' at all?

  0 Comments

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!