MATLAB Answers

Deep Q Learning - define an adaptive critic learning rate?

5 views (last 30 days)
Hello,
at the moment i use Deep Q Learning for process planning and i would like to use an adaptive critic learning rate to speed up the training.
Is there any direct way (or workaround) to use a learning rate that lowers over the training process, e.g. depending on the number of epochs/steps, in DQL?
Thanks in advance and best wishes
Niklas

  0 Comments

Sign in to comment.

Accepted Answer

Emmanouil Tzorakoleftherakis
Hi Niklas,
I believe this is currently not supported. This is an interesting usecase though - I will inform the development team. Is there any particular model you have in mind that would work better? For example linear/exponential decay, etc.

  7 Comments

Show 4 older comments
Emmanouil Tzorakoleftherakis
If for example, the IsDone flag is activated at the initial episodes of training, that would be fine (imagine stopping an episode if an agent violates some constraint). Eventually though, you would want the agent to learn how to respect these constraints and let the episode terminate naturally. If you, for whatever reason, manually and consistently stop/reduce the episode duration, you are reducing the potential maximum episode reward that can be collected. So, for a more "mature" agent, you may actually he inhibiting its learning potential, meaning that you may see the collected episode reward go down as you reduce the episode duration (which is kind of expected since there are less simulations steps, and thus less collected rewards). Hope that helps
Magnify
Magnify on 29 Jul 2020
There is one more question why the frequency of Agent's action outport is 0.05s rather than 0.025s specified by the agent sample time in my script createDDPGAgent.m, moreover, there is no way to modify it. there is a picture about sample time display as follow:sample time displayI would appreciate it if you give some tips to me.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!