rlACAgent

Create actor-critic reinforcement learning agent

Syntax

agent = rlACAgent(actor,critic,opt)

Description

agent = rlACAgent(actor,critic,opt) creates an actor-critic (AC) agent with the specified actor and critic networks, using the specified AC agent options. For more information on AC agents, see Actor-Critic Agents.

Examples

Create Actor-Critic Agent

Create an environment interface and obtain its observation and action specifications.

env = rlPredefinedEnv("CartPole-Discrete");
obsInfo = getObservationInfo(env);
actInfo = getActionInfo(env);

Create a critic representation.

criticNetwork = [
    imageInputLayer([4 1 1],'Normalization','none','Name','state')
    fullyConnectedLayer(1,'Name','CriticFC')];
criticOpts = rlRepresentationOptions('LearnRate',8e-3,'GradientThreshold',1);
critic = rlRepresentation(criticNetwork,obsInfo,'Observation',{'state'},criticOpts);

Create an actor representation.

actorNetwork = [
    imageInputLayer([4 1 1],'Normalization','none','Name','state')
    fullyConnectedLayer(2,'Name','action')];
actorOpts = rlRepresentationOptions('LearnRate',8e-3,'GradientThreshold',1);
actor = rlRepresentation(actorNetwork,obsInfo,actInfo,...
    'Observation',{'state'},'Action',{'action'},actorOpts);

Specify agent options, and create an AC agent using the environment, actor, and critic.

agentOpts = rlACAgentOptions(...
    'NumStepsToLookAhead',32, ...
    'DiscountFactor',0.99);
agent = rlACAgent(actor,critic,agentOpts);

Input Arguments

actor — Actor network representation

rlLayerRepresentation object

Actor network representation for representing the policy, specified as either an rlLayerRepresentation or rlDLNetworkRepresentation object created using rlRepresentation. For more information on creating actor representations, see Create Policy and Value Function Representations.

critic — Critic network representation

rlLayerRepresentation object

Critic network representation for estimating the state-value function, specified as an either an rlLayerRepresentation or rlDLNetworkRepresentation object created using rlRepresentation. For more information on creating critic representations, see Create Policy and Value Function Representations.

opt — Agent options

rlACAgentOptions object

Agent options, specified as an rlACAgentOptions object.

Output Arguments

agent — AC agent

rlACAgent object

AC agent, returned as an rlACAgent object.