Main Content

setLearnableParameters

Set learnable parameter values of agent, function approximator, or policy object

Description

Agent

setLearnableParameters(agent) sets the learnable parameter values specified in pars in the specified agent.

agent = setLearnableParameters(agent) also returns the new agent as an output argument.

Actor or Critic

example

newFcn = setLearnableParameters(oldFcn,pars) returns a new actor or critic function approximator object, newFcn, with the same structure as the original function object, oldFcn, and the learnable parameter values specified in pars.

Policy

newPol = setLearnableParameters(oldPol,pars) returns a new policy object, newPol, with the same structure as the original function object, oldFcn, and the learnable parameter values specified in pars.

Examples

collapse all

Assume that you have an existing trained reinforcement learning agent. For this example, load the trained agent from Compare DDPG Agent to LQR Controller.

load("DoubleIntegDDPG.mat","agent") 

Obtain the critic function approximator from the agent.

critic = getCritic(agent);

Obtain the learnable parameters from the critic.

params = getLearnableParameters(critic)
params=2×1 cell array
    {[-5.0182 -1.5718 -0.3493 -0.1067 -0.0540 -0.0029]}
    {[                                              0]}

Modify the parameter values. For this example, simply multiply all of the parameters by 2.

modifiedParams = cellfun(@(x) x*2,params,"UniformOutput",false);

Set the parameter values of the critic to the new modified values.

critic = setLearnableParameters(critic,modifiedParams);

Set the critic in the agent to the new modified critic.

setCritic(agent,critic);

Display the new parameter values.

getLearnableParameters(getCritic(agent))
ans=2×1 cell array
    {[-10.0364 -3.1436 -0.6987 -0.2135 -0.1080 -0.0059]}
    {[                                               0]}

Assume that you have an existing trained reinforcement learning agent. For this example, load the trained agent from Compare DDPG Agent to LQR Controller.

load("DoubleIntegDDPG.mat","agent") 

Obtain the actor function approximator from the agent.

actor = getActor(agent);

Obtain the learnable parameters from the actor.

params = getLearnableParameters(actor)
params=2×1 cell array
    {[-15.5717 -7.1444]}
    {[               0]}

Modify the parameter values. For this example, simply multiply all of the parameters by 2.

modifiedParams = cellfun(@(x) x*2,params,"UniformOutput",false);

Set the parameter values of the actor to the new modified values.

actor = setLearnableParameters(actor,modifiedParams);

Set the actor in the agent to the new modified actor.

setActor(agent,actor);

Display the new parameter values.

getLearnableParameters(getActor(agent))
ans=2×1 cell array
    {[-31.1433 -14.2887]}
    {[                0]}

Input Arguments

collapse all

Reinforcement learning agent, specified as one of the following objects:

Note

agent is an handle object. Therefore its parameters are updated by setLearnableParameters whether agent is returned as an output argument or not. For more information about handle objects, see Handle Object Behavior.

Original function approximator object, specified as one of the following:

To create an actor or critic function object, use one of the following methods.

  • Create a function object directly.

  • Obtain the existing critic from an agent using getCritic.

  • Obtain the existing actor from an agent using getActor.

Reinforcement learning policy, specified as one of the following objects:

Learnable parameter values for the representation object, specified as a cell array. The parameters in pars must be compatible with the structure and parameterization of the agent, function approximator, or policy object passed as a first argument.

To obtain a cell array of learnable parameter values from an existing agent, function approximator, or policy object , which you can then modify, use the getLearnableParameters function.

Output Arguments

collapse all

New actor or critic object, returned as a function object of the same type as oldFcn. Except of its new learnable parameter values, newFcn is the same as oldFcn.

New reinforcement learning policy, returned as a policy object of the same type as oldPol. Apart from the learnable parameter values, newPol is the same as oldPol.

Updated agent, returned as an agent object. Note that agent is an handle object. Therefore its parameters are updated by setLearnableParameters whether agent is returned as an output argument or not. For more information about handle objects, see Handle Object Behavior.

Version History

Introduced in R2019a

expand all