Use trained RL agent to predit output on random test sample

This is my code

  Required_agent=load('G1_Agents.mat', 'agent1', 'agent2','agent3');
        test_sample=final_test_data_rearranged;
        final_test_sample = reshape(test_sample, [], 1);
        fprintf('Required_sample = %d\n', i)
        agent1 = Required_agent.agent1;
        agent2 = Required_agent.agent2;
        agent3 = Required_agent.agent3;
        policy1=generatePolicyFunction(agent1);
        policy2=generatePolicyFunction(agent2);
        policy3=generatePolicyFunction(agent3);
        action1 = getAction(policy1,final_test_sample);
        action2 = getAction(policy2,final_test_sample);
        action3 = getAction(policy3,final_test_sample);

When I run this code, the error is:

Incorrect number or types of inputs or outputs for function getAction.

action1 = getAction(policy1,final_test_sample);

I have cross-checked the final_test_sample; it has the same size as the input used during the training of the RL agent.

[Walter Roberson]

Note that the second input to getAction must be a cell array. We cannot tell from your code what data type final_test_sample is.

[Hitesh]

Hi @Praveen Verma,

Could you share the complete code or the file so that we reproduce the error and can further look for root cause of the error ?

Answer 1

[Aravind]

Hello @Praveen Verma,

It seems you are experiencing an error related to input or output types for the "getAction" function. After reviewing your code, it appears that the issue stems from how the input arguments are being used, particularly the first argument, which is the policy intended for action calculation.

The "policy1" variable is derived from the "generatePolicyFunction" function. According to the official documentation at https://www.mathworks.com/help/reinforcement-learning/ref/rl.policy.rlmaxqpolicy.generatepolicyfunction.html, "generatePolicyFunction" does not return an output argument. Instead, it creates an "evaluatePolicy.m" file containing the policy function and an "agentData.mat" file with the trained deep neural network actor.

If you attempt to assign the output of "generatePolicyFunction" to a variable, that variable will contain the content intended for the "evaluatePolicy.m" file as a string. As per the documentation here, the input to the "getAction" function should be a policy object, not a string, which is what your code currently passes, causing the error.

To resolve this, I suggest using the agent object directly as input to the "getAction" function instead of a policy object. Alternatively, you can use the "evaluatePolicy" function from the "evaluatePolicy.m" file to compute the required action.

Here are some examples to guide you in calculating an action from a trained RL agent:

1. Obtaining action directly using the agent object: https://www.mathworks.com/help/reinforcement-learning/ref/rl.policy.rlmaxqpolicy.getaction.html#mw_c350da09-b464-40ca-90cb-1d790cdec905
2. Using the "generatePolicyFunction" function to calculate the action: https://www.mathworks.com/help/reinforcement-learning/ref/rl.policy.rlmaxqpolicy.generatepolicyfunction.html#mw_1e55e0cb-2397-4f7d-be10-30d252b6215f
3. Obtaining an action using the policy object: https://www.mathworks.com/help/reinforcement-learning/ref/rl.policy.rlmaxqpolicy.getaction.html#mw_2253cbf8-d771-409a-b01a-ab1a1f30eec9

I hope this helps resolve your issue.