generatePolicyFunction

Create function that evaluates trained policy of reinforcement learning agent

Syntax

generatePolicyFunction(agent)

generatePolicyFunction(agent,Name,Value)

Description

generatePolicyFunction(agent) creates a function that evaluates the learned policy of the specified agent using the default function name, policy name, and data file name. After generating the policy evaluation function, you can:

• Generate code for the function using MATLAB^® Coder™ or GPU Coder™. For more information, see Deploy Trained Reinforcement Learning Policies.

• Simulate the trained agent in Simulink^® using a MATLAB Function (Simulink) block.

generatePolicyFunction(agent,Name,Value) specifies the function name, policy name, and data file name using one or more name-value pair arguments.

Examples

Create Policy Evaluation Function for PG Agent

This example shows how to create a policy evaluation function for a PG Agent.

First, create and train a reinforcement learning agent. For this example, load the PG agent trained in Train PG Agent to Balance Cart-Pole System.

load('MATLABCartpolePG.mat','agent')

Then, create a policy evaluation function for this agent using default names.

generatePolicyFunction(agent);

This command creates the evaluatePolicy.m file, which contains the policy function, and the agentData.mat file, which contains the trained deep neural network actor.

View the generated function.

type evaluatePolicy.m

function action1 = evaluatePolicy(observation1)
%#codegen

% Reinforcement Learning Toolbox
% Generated on: 20-Jan-2022 15:07:48

persistent policy;
if isempty(policy)
	policy = coder.loadRLPolicy("agentData.mat");
end
% evaluate the policy
action1 = getAction(policy,observation1);

Evaluate the policy for a random observation.

evaluatePolicy(rand(4,1))

ans = 10

You can now generate code for this policy function using MATLAB® Coder™. For more information, see Deploy Trained Reinforcement Learning Policies.

Create Policy Evaluation Function for Q-Learning Agent

This example shows how to create a policy evaluation function for a Q-Learning Agent.

For this example, load the Q-learning agent trained in Train Reinforcement Learning Agent in Basic Grid World

load('basicGWQAgent.mat','qAgent')

Create a policy evaluation function for this agent and specify the name of the agent data file.

generatePolicyFunction(qAgent,'MATFileName',"policyFile.mat")

This command creates the evaluatePolicy.m file, which contains the policy function, and the policyFile.mat file, which contains the trained Q table value function.

View the generated function.

type evaluatePolicy.m

function action1 = evaluatePolicy(observation1)
%#codegen

% Reinforcement Learning Toolbox
% Generated on: 20-Jan-2022 15:24:34

persistent policy;
if isempty(policy)
	policy = coder.loadRLPolicy("policyFile.mat");
end
% evaluate the policy
action1 = getAction(policy,observation1);

Evaluate the policy for a random observation.

evaluatePolicy(randi(25))

ans = 1

You can now generate code for this policy function using MATLAB® Coder™. For more information, see Deploy Trained Reinforcement Learning Policies.

Input Arguments

agent — Reinforcement learning agent

reinforcement learning agent object

Trained reinforcement learning agent, specified as one of the following agent objects. To train your agent, use the train function.

• rlQAgent

• rlSARSAAgent

• rlDQNAgent

• rlDDPGAgent

• rlTD3Agent

• rlACAgent

• rlPGAgent

• rlPPOAgent

• rlTRPOAgent

• rlSACAgent

For agents with a stochastic actor (PG, PPO, SAC, TRPO, AC), the action returned by the generated policy function depends on the value of the UseExplorationPolicy property of the agent. By default, UseExplorationPolicy is false and the generated action is deterministic. If UseExplorationPolicy is true, the generated action is stochastic.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: 'FunctionName',"computeAction"

FunctionName — Name of the generated function

"evaluatePolicy" (default) | string | character vector

Name of the generated function, specified as a string or character vector.

PolicyName — Name of the policy variable within the generated function

"policy" (default) | string | character vector

Name of the policy variable within the generated function, specified as a string or character vector.

MATFileName — Name of agent data file

"agentData" (default) | string | character vector

Name of the agent data file, specified as a string or character vector.

Version History

Introduced in R2019a

R2022a: Code generated by generatePolicyFunction now uses policy objects

Behavior change in future release

The code generated by generatePolicyFunction now loads a deployable policy object from a reinforcement learning agent. The results from running the generated policy function remain the same.