MATLAB Answers

How can I extract a trained RL Agent's network's weights and biases?

10 views (last 30 days)
rakbar
rakbar on 26 Mar 2020
Commented: rakbar on 29 Mar 2020
How can I extract a trained RL Agent's network's weights and biases?
My network is:
statePath = [
imageInputLayer([numObservations 1 1], 'Normalization', 'none', 'Name', 'state')
fullyConnectedLayer(NumNeuron, 'Name', 'CriticStateFC1')
reluLayer('Name', 'CriticRelu1')
fullyConnectedLayer(NumNeuron, 'Name', 'CriticStateFC2')];
actionPath = [
imageInputLayer([1 1 1], 'Normalization', 'none', 'Name', 'action')
fullyConnectedLayer(NumNeuron, 'Name', 'CriticActionFC1')
reluLayer('Name', 'ActorRelu1')
fullyConnectedLayer(NumNeuron, 'Name', 'CriticActionFC2')];
commonPath = [
additionLayer(2,'Name', 'add')
reluLayer('Name','CriticCommonRelu')
fullyConnectedLayer(1, 'Name', 'output')];
criticNetwork = layerGraph(statePath);
criticNetwork = addLayers(criticNetwork, actionPath);
criticNetwork = addLayers(criticNetwork, commonPath);
criticNetwork = connectLayers(criticNetwork,'CriticStateFC2','add/in1');
criticNetwork = connectLayers(criticNetwork,'CriticActionFC2','add/in2');
% set some options for the critic
criticOpts = rlRepresentationOptions('LearnRate',learing_rate,...
'GradientThreshold',1);
% create the critic based on the network approximator
critic = rlQValueRepresentation(criticNetwork,obsInfo,actInfo,...
'Observation',{'state'},'Action',{'action'},criticOpts);
agent = rlDQNAgent(critic,agentOpts)
trainingStats = train(agent,env,trainOpts);
After training, I'd like to get the network's trained weights and biases.

  0 Comments

Sign in to comment.

Accepted Answer

Anh Tran
Anh Tran on 27 Mar 2020
Edited: Anh Tran on 27 Mar 2020
You can get the parameters from the trained's critic representation for DQN agent. In MATLAB R2020a, see getLearnableParameters and getCritic functions (function name changes a bit since R2019b). You can follow similar steps to get the actor's parameters from actor-based agent like DDPG or PPO.
critic = getCritic(agent);
criticParams = getLearnableParameters(critic);

  2 Comments

rakbar
rakbar on 28 Mar 2020
Great, thank you! I didn't realize this was in the documentation already.
Thanks!
rakbar
rakbar on 29 Mar 2020
Is there a way to get an Agent's weights and bias after each training episode?

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!