If you create a custom deep learning layer, then you can use
the checkLayer
function
to check that the layer is valid. The function checks layers for validity, GPU compatibility,
correctly defined gradients, and code generation compatibility. To check that a layer is valid,
run the following
command:
checkLayer(layer,validInputSize,'ObservationDimension',dim)
layer
is an instance of the layer, validInputSize
is a
vector or cell array specifying the valid input sizes to the layer, and dim
specifies the dimension of the observations in the layer input data. For large input sizes, the gradient checks take longer to run. To speed up the tests, specify a smaller valid input size.Check the validity of the example custom layer preluLayer
.
Define a custom PReLU layer. To create this layer, save the file preluLayer.m
in the current folder.
Create an instance of the layer and check that it is valid using checkLayer
. Set the valid input size to the typical size of a single observation input to the layer. For a single input, the layer expects observations of size hbywbyc, where h, w, and c are the height, width, and number of channels of the previous layer output, respectively.
Specify validInputSize
as the typical size of an input array.
layer = preluLayer(20,'prelu');
validInputSize = [5 5 20];
checkLayer(layer,validInputSize)
Skipping multiobservation tests. To enable tests with multiple observations, specify the 'ObservationDimension' option. For 2D image data, set 'ObservationDimension' to 4. For 3D image data, set 'ObservationDimension' to 5. For sequence data, set 'ObservationDimension' to 2. Skipping GPU tests. No compatible GPU device found. Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options. Running nnet.checklayer.TestLayerWithoutBackward ......... Done nnet.checklayer.TestLayerWithoutBackward __________ Test Summary: 9 Passed, 0 Failed, 0 Incomplete, 13 Skipped. Time elapsed: 0.14162 seconds.
The results show the number of passed, failed, and skipped tests. If you do not specify the 'ObservationsDimension'
option, or do not have a GPU, then the function skips the corresponding tests.
Check Multiple Observations
For multiobservation input, the layer expects an array of observations of size hbywbycbyN, where h, w, and c are the height, width, and number of channels, respectively, and N is the number of observations.
To check the layer validity for multiple observations, specify the typical size of an observation and set 'ObservationDimension'
to 4.
layer = preluLayer(20,'prelu'); validInputSize = [5 5 20]; checkLayer(layer,validInputSize,'ObservationDimension',4)
Skipping GPU tests. No compatible GPU device found. Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options. Running nnet.checklayer.TestLayerWithoutBackward .......... ... Done nnet.checklayer.TestLayerWithoutBackward __________ Test Summary: 13 Passed, 0 Failed, 0 Incomplete, 9 Skipped. Time elapsed: 0.064861 seconds.
In this case, the function does not detect any issues with the layer.
The checkLayer
function checks the validity of a custom layer by performing a series of tests.
The checkLayer
function uses these tests to check the validity of custom
intermediate layers (layers of type nnet.layer.Layer
).
Test  Description 

functionSyntaxesAreCorrect  The syntaxes of the layer functions are correctly defined. 
predictDoesNotError  predict does not error. 
forwardDoesNotError  When specified, 
forwardPredictAreConsistentInSize  When 
backwardDoesNotError  When specified, backward does not error. 
backwardIsConsistentInSize  When

predictIsConsistentInType  The outputs of 
forwardIsConsistentInType  When 
backwardIsConsistentInType  When 
gradientsAreNumericallyCorrect  When backward is specified, the gradients computed
in backward are consistent with the numerical
gradients. 
backwardPropagationDoesNotError  When backward is not specified, the derivatives
can be computed using automatic differentiation. 
codegenPragmaDefinedInClassDef  The pragma "%#codegen" for code generation is
specified in class file. 
checkForSupportedLayerPropertiesForCodegen  The layer properties support code generation. 
predictIsValidForCodeGeneration  predict is valid for code generation. 
The tests predictIsConsistentInType
, forwardIsConsistentInType
, and backwardIsConsistentInType
also check for GPU compatibility. To execute the layer functions on a GPU, the functions must support inputs and outputs of type gpuArray
with the underlying data type single
.
The checkLayer
function uses these tests to check the
validity of custom output layers (layers of type
nnet.layer.ClassificationLayer
or
nnet.layer.RegressionLayer
).
Test  Description 

forwardLossDoesNotError  forwardLoss does not error. 
backwardLossDoesNotError  backwardLoss does not error. 
forwardLossIsScalar  The output of forwardLoss is scalar. 
backwardLossIsConsistentInSize  When backwardLoss is specified, the output of
backwardLoss is consistent in size:
dLdY is the same size as the predictions
Y . 
forwardLossIsConsistentInType  The output of 
backwardLossIsConsistentInType  When 
gradientsAreNumericallyCorrect  When backwardLoss is specified, the gradients computed
in backwardLoss are numerically correct. 
backwardPropagationDoesNotError  When backwardLoss is not specified, the derivatives
can be computed using automatic differentiation. 
The forwardLossIsConsistentInType
and
backwardLossIsConsistentInType
tests also check for GPU compatibility. To
execute the layer functions on a GPU, the functions must support inputs and outputs of type
gpuArray
with the underlying data type single
.
To check the layer validity, the checkLayer
function generates data depending on the type of layer:
Layer Type  Description of Generated Data 

Intermediate  Values in the range [1,1] 
Regression output  Predictions and targets with values in the range [1,1] 
Classification output  Predictions with values in the range [0,1]. If
you specify the If you do
not specify the 
To check for multiple observations, specify the observation dimension using the 'ObservationDimension'
namevalue pair. If you specify the observation dimension, then the checkLayer
function checks that the layer functions are valid using generated data with minibatches of size 1 and 2. If you do not specify this namevalue pair, then the function skips the tests that check that the layer functions are valid for multiple observations.
If a test fails when you use checkLayer
,
then the function provides a test diagnostic and a framework diagnostic. The test
diagnostic highlights any issues found with the layer. The framework diagnostic provides
more detailed information.
The test functionSyntaxesAreCorrect
checks that the layer
functions have correctly defined syntaxes.
Test Diagnostic  Description  Possible Solution 

Incorrect number of input arguments for 'predict' in
Layer .  The syntax for the predict function is not
consistent with the number of layer inputs.  Specify the correct number of input and output
arguments in The syntax for Tip If the number of inputs to Tip If the custom layer has a 
Incorrect number of output arguments for 'predict' in
Layer  The syntax for the predict function is not
consistent with the number of layer outputs.  
Incorrect number of input arguments for 'forward' in
Layer  The syntax for the optional forward function
is not consistent with the number of layer inputs.  Specify the correct number of input and output
arguments in The syntax for Tip If the number of inputs to Tip If the custom layer has a 
Incorrect number of output arguments for 'forward' in
Layer  The syntax for the optional forward function
is not consistent with the number of layer outputs.  
Incorrect number of input arguments for 'backward' in
Layer  The syntax for the optional backward function
is not consistent with the number of layer inputs and
outputs.  Specify the correct number of input and output
arguments in The syntax for
For the outputs, Tip If the number of inputs to If the number of outputs can vary, then use Tip If the layer forward functions support 
Incorrect number of output arguments for 'backward' in
Layer  The syntax for the optional backward function
is not consistent with the number of layer outputs. 
For layers with multiple inputs or outputs, you must set the values of the layer
properties NumInputs
(or alternatively,
InputNames
) and NumOutputs
(or
alternatively, OutputNames
) in the layer constructor function,
respectively.
The checkLayer
function checks that the layer functions are
valid for single and multiple observations. To check for multiple observations, specify the observation dimension using the 'ObservationDimension'
namevalue pair. If you specify the observation dimension, then the checkLayer
function checks that the layer functions are valid using generated data with minibatches of size 1 and 2. If you do not specify this namevalue pair, then the function skips the tests that check that the layer functions are valid for multiple observations.
Test Diagnostic  Description  Possible Solution 

Skipping multiobservation tests. To enable checks with
multiple observations, specify the 'ObservationDimension'
parameter in checkLayer .  If you do not specify the
'ObservationDimension' parameter in
checkLayer , then the function skips the
tests that check data with multiple observations.  Use the command
For more information, see Layer Input Sizes. 
These tests check that the layers do not error when passed input data of valid size.
Intermediate Layers. The tests predictDoesNotError
,
forwardDoesNotError
, and
backwardDoesNotError
check that the layer functions do
not error when passed inputs of valid size. If you specify an observation
dimension, then the function checks the layer for both a single observation and
multiple observations.
Test Diagnostic  Description  Possible Solution 

The function 'predict' threw an
error:  The predict function errors when
passed data of size
validInputSize .  Address the error described in the
Tip If the layer forward functions support 
The function 'forward' threw an
error:  The optional forward function errors
when passed data of size
validInputSize .  
The function 'backward' threw an
error:  The optional backward function errors
when passed the output of
predict . 
Output Layers. The tests forwardLossDoesNotError
and
backwardLossDoesNotError
check that the layer functions
do not error when passed inputs of valid size. If you specify an observation
dimension, then the function checks the layer for both a single observation and
multiple observations.
Test Diagnostic  Description  Possible Solution 

The function 'forwardLoss' threw an
error:  The forwardLoss function errors when
passed data of size
validInputSize .  Address the error described in the
Tip If the 
The function 'backwardLoss' threw an
error:  The optional backwardLoss function
errors when passed data of size
validInputSize . 
These tests check that the layer function outputs are consistent in size.
Intermediate Layers. The test backwardIsConsistentInSize
checks that the
backward
function outputs derivatives of the correct
size.
The syntax for backward
is [dLdX1,…,dLdXn,dLdW1,…,dLdWk] =
backward(layer,X1,…,Xn,Z1,…,Zm,dLdZ1,…,dLdZm,memory)
, where:
X1,…,Xn
are the n
layer inputs
Z1,…,Zm
are the m
outputs of the layer
forward functions
dLdZ1,…,dLdZm
are the gradients backward propagated from
the next layer
memory
is the memory output of forward
if forward
is defined, otherwise, memory
is []
.
For the outputs, dLdX1,…,dLdXn
are the derivatives of the
loss with respect to the layer inputs and dLdW1,…,dLdWk
are the
derivatives of the loss with respect to the k
learnable parameters. To
reduce memory usage by preventing unused variables being saved between the forward and
backward pass, replace the corresponding input arguments with ~
.
Tip
If the number of inputs to backward
can vary, then use
varargin
instead of the input arguments after
layer
. In this case, varargin
is a cell array
of the inputs, where varargin{i}
corresponds to Xi
for i
=1,…,NumInputs
,
varargin{NumInputs+j}
and
varargin{NumInputs+NumOutputs+j}
correspond to
Zj
and dLdZj
, respectively, for
j
=1,…,NumOutputs
, and
varargin{end}
corresponds to memory
.
If the number of outputs can vary, then use varargout
instead of the
output arguments. In this case, varargout
is a cell array of the
outputs, where varargout{i}
corresponds to dLdXi
for i
=1,…,NumInputs
and
varargout{NumInputs+t}
corresponds to dLdWt
for t
=1,…,k
, where k
is the
number of learnable parameters.
The derivatives dLdX1
, …, dLdXn
must be
the same size as the corresponding layer inputs, and
dLdW1,…,dLdWk
must be the same size as the corresponding
learnable parameters. The sizes must be consistent for input data with single
and multiple observations.
Test Diagnostic  Description  Possible Solution 

Incorrect size of 'dLdX' for
'backward' .  The derivatives of the loss with respect to the layer inputs must be the same size as the corresponding layer input.  Return the derivatives

Incorrect size of the derivative of the loss
with respect to the input 'in1' for
'backward'  
The size of 'Z' returned from 'forward' must be
the same as for 'predict' .  The outputs of predict must be the
same size as the corresponding outputs of
forward .  Return the outputs 
Incorrect size of the derivative of the loss
with respect to 'W' for 'backward' .  The derivatives of the loss with respect to the learnable parameters must be the same size as the corresponding learnable parameters.  Return the derivatives

Tip
If the layer forward functions support dlarray
objects, then the software automatically determines the backward function and you do not need to specify the backward
function. For a list of functions that support dlarray
objects, see List of Functions with dlarray Support.
Output Layers. The test forwardLossIsScalar
checks that the output of the
forwardLoss
function is scalar. When the
backwardLoss
function is specified, the test
backwardLossIsConsistentInSize
checks that the outputs of
forwardLoss
and backwardLoss
are of
the correct size.
The syntax for forwardLoss
is loss
= forwardLoss(layer, Y, T)
. The input Y
corresponds to the
predictions made by the network. These predictions are the output of the previous layer. The
input T
corresponds to the training targets. The output
loss
is the loss between Y
and T
according to the specified loss function. The output loss
must be
scalar.
If the forwardLoss
function supports
dlarray
objects, then the software automatically determines
the backward loss function and you do not need to specify the
backwardLoss
function. For a list of functions that
support dlarray
objects, see List of Functions with dlarray Support.
The syntax for backwardLoss
is dLdY
= backwardLoss(layer, Y, T)
. The input Y
contains the
predictions made by the network and T
contains the training targets. The
output dLdY
is the derivative of the loss with respect to the predictions
Y
. The output dLdY
must be the same size as the layer
input Y
.
Test Diagnostic  Description  Possible Solution 

Incorrect size of 'loss' for
'forwardLoss' .  The output loss of
forwardLoss must be a scalar.  Return the output 
Incorrect size of the derivative of loss 'dLdY'
for 'backwardLoss' .  When backwardLoss is specified, the
derivatives of the loss with respect to the layer input must
be the same size as the layer input.  Return derivative If the

These tests check that the layer function outputs are consistent in type and that the layer functions are GPU compatible.
If the layer forward functions fully support dlarray
objects, then the layer
is GPU compatible. Otherwise, to be GPU compatible, the layer functions must support inputs
and return outputs of type gpuArray
(Parallel Computing Toolbox).
Many MATLAB^{®} builtin functions support gpuArray
(Parallel Computing Toolbox) and dlarray
input arguments. For a list of
functions that support dlarray
objects, see List of Functions with dlarray Support. For a list of functions
that execute on a GPU, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
To use a GPU for deep
learning, you must also have a supported GPU device. For information on supported devices, see
GPU Support by Release (Parallel Computing Toolbox). For more information on working with GPUs in MATLAB, see GPU Computing in MATLAB (Parallel Computing Toolbox).
Intermediate Layers. The tests predictIsConsistentInType
,
forwardIsConsistentInType
, and
backwardIsConsistentInType
check that the layer functions
output variables of the correct data type. The tests check that the layer
functions return consistent data types when given inputs of the data types
single
, double
, and
gpuArray
with the underlying types
single
or double
.
Tip
If you preallocate arrays using functions such as
zeros
, then you must ensure that the data types of these arrays are
consistent with the layer function inputs. To create an array of zeros of the same data type as
another array, use the 'like'
option of zeros
. For
example, to initialize an array of zeros of size sz
with the same data type
as the array X
, use Z = zeros(sz,'like',X)
.
Test Diagnostic  Description  Possible Solution 

Incorrect type of 'Z' for
'predict' .  The types of the outputs
Z1,…,Zm of the
predict function must be consistent
with the inputs X1,…,Xn .  Return the outputs

Incorrect type of output 'out1' for
'predict' .  
Incorrect type of 'Z' for
'forward' .  The types of the outputs
Z1,…,Zm of the optional
forward function must be consistent
with the inputs X1,…,Xn .  
Incorrect type of output 'out1' for
'forward' .  
Incorrect type of 'dLdX' for
'backward' .  The types of the derivatives
dLdX1,…,dLdXn of the optional
backward function must be consistent
with the inputs X1,…,Xn .  Return the derivatives

Incorrect type of the derivative of the loss
with respect to the input 'in1' for
'backward' .  
Incorrect type of the derivative of loss with
respect to 'W' for 'backward' .  The type of the derivative of the loss of the learnable parameters must be consistent with the corresponding learnable parameters.  For each learnable parameter, return the derivative with the same type as the corresponding learnable parameter. 
Tip
If the layer forward functions support dlarray
objects, then the software automatically determines the backward function and you do not need to specify the backward
function. For a list of functions that support dlarray
objects, see List of Functions with dlarray Support.
Output Layers. The tests forwardLossIsConsistentInType
and
backwardLossIsConsistentInType
check that the layer
functions output variables of the correct data type. The tests check that the
layers return consistent data types when given inputs of the data types
single
, double
, and
gpuArray
with the underlying types
single
or double
.
Test Diagnostic  Description  Possible Solution 

Incorrect type of 'loss' for
'forwardLoss' .  The type of the output loss of the
forwardLoss function must be
consistent with the input Y .  Return 
Incorrect type of the derivative of loss 'dLdY'
for 'backwardLoss' .  The type of the output dLdY of the
optional backwardLoss function must be
consistent with the input Y .  Return 
Tip
If the forwardLoss
function supports dlarray
objects, then the software automatically determines the backward loss function and you do not need to specify the backwardLoss
function. For a list of functions that support dlarray
objects, see List of Functions with dlarray Support.
The test gradientsAreNumericallyCorrect
checks that the
gradients computed by the layer functions are numerically correct. The test
backwardPropagationDoesNotError
checks that the derivatives
can be computed using automatic differentiation.
Intermediate Layers. When the optional backward
function is not specified, the
test backwardPropagationDoesNotError
checks that the
derivatives can be computed using automatic differentiation. When the optional
backward
function is specified, the test
gradientsAreNumericallyCorrect
tests that the gradients
computed in backward
are numerically correct.
Test Diagnostic  Description  Possible Solution 

Expected a dlarray with no dimension labels, but
instead found labels .  When the optional backward function is
not specified, the layer forward functions must output
dlarray objects without dimension
labels.  Ensure that any dlarray objects created
in the layer forward functions do not contain dimension
labels. 
Unable to backward propagate through the layer.
Check that the 'forward' function fully supports
automatic differentiation. Alternatively, implement the
'backward' function manually .  One or more of the following:
 Check that the forward functions
support Check that the derivatives of the input
Alternatively, define a custom backward
function by creating a function named

Unable to backward propagate through the layer.
Check that the 'predict' function fully supports
automatic differentiation. Alternatively, implement the
'backward' function manually .  
The derivative 'dLdX' for 'backward' is
inconsistent with the numerical
gradient .  One or more of the following:
 If the layer forward functions support
Check that the derivatives in
If the derivatives are correctly
computed, then in the If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic. 
The derivative of the loss with respect to the
input 'in1' for 'backward' is inconsistent with the
numerical gradient .  
The derivative of loss with respect to 'W' for
'backward' is inconsistent with the numerical
gradient . 
Tip
If the layer forward functions support dlarray
objects, then the software automatically determines the backward function and you do not need to specify the backward
function. For a list of functions that support dlarray
objects, see List of Functions with dlarray Support.
Output Layers. When the optional backwardLoss
function is not specified,
the test backwardPropagationDoesNotError
checks that the
derivatives can be computed using automatic differentiation. When the optional
backwardLoss
function is specified, the test
gradientsAreNumericallyCorrect
tests that the gradients
computed in backwardLoss
are numerically correct.
Test Diagnostic  Description  Possible Solution 

Expected a dlarray with no dimension labels, but
instead found labels  When the optional backwardLoss
function is not specified, the
forwardLoss function must output
dlarray objects without dimension
labels.  Ensure that any dlarray objects created
in the forwardLoss function does not
contain dimension labels. 
Unable to backward propagate through the layer.
Check that the 'forwardLoss' function fully supports
automatic differentiation. Alternatively, implement the
'backwardLoss' function manually  One or more of the following:
 Check that the Check that the derivatives of the input
Alternatively, define a custom backward
loss function by creating a function named

The derivative 'dLdY' for 'backwardLoss' is
inconsistent with the numerical
gradient .  One or more of the following:
 Check that the derivatives in
If the derivatives are correctly
computed, then in the If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic. 
Tip
If the forwardLoss
function supports dlarray
objects, then the software automatically determines the backward loss function and you do not need to specify the backwardLoss
function. For a list of functions that support dlarray
objects, see List of Functions with dlarray Support.
If you set the 'CheckCodegenCompatibility'
option to
true
, then the checkLayer
function
checks the layer for code generation compatibility.
The test codegenPragmaDefinedInClassDef
checks that the layer
definition contains the code generation pragma %#codegen
. The
test checkForSupportedLayerPropertiesForCodegen
checks that the
layer properties support code generation. The test
predictIsValidForCodegeneration
checks that the outputs of
predict
are consistent in dimension and batch size.
Code generation supports intermediate layers with 2D image input only.
Test Diagnostic  Description  Possible Solution 

Specify '%#codegen' in the class definition of
custom layer  The layer definition does not include the pragma
"%#codegen" for code generation. 
Add the 
Nonscalar layer properties must be type single or
double or character array for custom layer  The layer contains nonscalar properties of type other than single, double, or character array.  Convert nonscalar properties to use a representation of type single, double, or character array. For
example, convert a categorical array to an array of integers
of type 
Scalar layer properties must be numeric, logical, or
string for custom layer  The layer contains scalar properties of type other than numeric, logical, or string.  Convert scalar properties to use a numeric representation, or a representation of type logical or string. For example, convert a categorical
scalar to an integer of type 
For code generation, 'Z' must have the same number
of dimensions as the layer input .  The number of dimensions of the output
 In the 
For code generation, 'Z' must have the same batch
size as the layer input .  The size of the batch size of the output
 In the 