How can I generate code for my deep neural network with ScalingLayer?

I am using GPU Coder in R2019b to generate CUDA code for my deep neural network policies following the workflow outlined here:

https://www.mathworks.com/help/reinforcement-learning/ug/deploy-trained-reinforcement-learning-agents.html#mw_f2a42f1d-5e31-4ac5-9d51-42d66da8d713

I get the following error:

"Error generating code for network policy_0. Code generation for ScalingLayer ActorScaling is not supported."

Is there a workaround for this unsupported layer?

Answer 1

[MathWorks Support Team]

The workaround is to substitute the 'ScalingLayer' with an 'nnet.onnx.layer.ElementwiseAffineLayer'. However, this would involve using a attached p-coded script ('getNetworkFromRepresentation.p') to extract the network from the 'rlLayerRepresentation' object. The steps are outlined in the script below:

%%load agent MAT file
load agent

%%get actor representation (contains scalingLayer) from saved agent
actor = getActor(saved_agent);

%%get the new network with codegen-supported layer
actorNetwork = getNetworkFromRepresentation(actor);
newActorNetwork = actorNetwork;

% replace any scalingLayer with nnet.onnx.layer.ElementwiseAffineLayer. The
% name, scale, bias values of old scalingLayer are transfered to the new
% layer
for ct = 1:numel(newActorNetwork.Layers)
    if isa(newActorNetwork.Layers(ct),'rl.layer.ScalingLayer')
        newLayer = nnet.onnx.layer.ElementwiseAffineLayer(...
            newActorNetwork.Layers(ct).Name,...
            newActorNetwork.Layers(ct).Scale,...
            newActorNetwork.Layers(ct).Bias);
        newActorNetwork = replaceLayer(newActorNetwork,newActorNetwork.Layers(ct).Name,newLayer);
    end
end

%%recreate the representation with net network
actionLayerName      = newActorNetwork.Layers(end).Name;
observationLayerName = newActorNetwork.Layers(1).Name;
actionInfo      = actor.ActionInfo;
observationInfo = actor.ObservationInfo;
newActor = rlRepresentation(newActorNetwork,observationInfo,actionInfo,...
    'Observation',observationLayerName,'Action',actionLayerName,actor.Options);
saved_agent = setActor(saved_agent,newActor);

%%codegen
generatePolicyFunction(saved_agent)

cfg = coder.gpuConfig('mex');
cfg.TargetLang = 'C++';
cfg.GpuConfig.Enabled = true;
cfg.DeepLearningConfig = coder.DeepLearningConfig('cudnn');
argstr = '{ones(4,1)}';

codegen('-config','cfg','evaluatePolicy','-args',argstr,'-report');

Notes:

• You are free to use the 'getNetworkFromRepresentation.p' script as you wish.
• You will need to install the 'Deep Learning Toolbox Converter for ONNX Model Format support package'.