Main Content

leakyrelu

Apply leaky rectified linear unit activation

Since R2019b

Description

The leaky rectified linear unit (ReLU) activation operation performs a nonlinear threshold operation, where any input value less than zero is multiplied by a fixed scale factor.

This operation is equivalent to

f(x)={x,x0scale*x,x<0.

Note

This function applies the leaky ReLU operation to dlarray data. If you want to apply leaky ReLU activation within a layerGraph object or Layer array, use the following layer:

example

Y = leakyrelu(X) computes the leaky ReLU activation of the input X by applying a threshold operation. All values in X less than zero are multiplied by a default scale factor of 0.01.

example

Y = leakyrelu(X,scaleFactor) specifies the scale factor for the leaky ReLU operation.

Examples

collapse all

Create a formatted dlarray object containing a batch of 128 28-by-28 images with 3 channels. Specify the format 'SSCB' (spatial, spatial, channel, batch).

miniBatchSize = 128;
inputSize = [28 28];
numChannels = 3;
X = rand(inputSize(1),inputSize(2),numChannels,miniBatchSize);
X = dlarray(X,"SSCB");

View the size and format of the input data.

size(X)
ans = 1×4

    28    28     3   128

dims(X)
ans = 
'SSCB'

Apply the leaky ReLU operation using the leakyrelu function.

Y = leakyrelu(X);

View the size and format of the output.

size(Y)
ans = 1×4

    28    28     3   128

dims(Y)
ans = 
'SSCB'

Create a formatted dlarray object containing a batch of 128 28-by-28 images with 3 channels. Specify the format 'SSCB' (spatial, spatial, channel, batch).

miniBatchSize = 128;
inputSize = [28 28];
numChannels = 3;
X = rand(inputSize(1),inputSize(2),numChannels,miniBatchSize);
X = dlarray(X,"SSCB");

View the size and format of the input data.

size(X)
ans = 1×4

    28    28     3   128

dims(X)
ans = 
'SSCB'

Apply the leaky ReLU operation using the leakyrelu function and specify a scale of 0.5.

Y = leakyrelu(X,0.5);

View the size and format of the output.

size(Y)
ans = 1×4

    28    28     3   128

dims(Y)
ans = 
'SSCB'

Input Arguments

collapse all

Input data, specified as a formatted or unformatted dlarray object.

Scale factor for negative inputs, specified as a numeric scalar. The default value is 0.01.

Data Types: single | double

Output Arguments

collapse all

Leaky ReLU activations, returned as a dlarray. The output Y has the same underlying data type as the input X.

If the input data X is a formatted dlarray, Y has the same dimension format as X. If the input data is not a formatted dlarray, Y is an unformatted dlarray with the same dimension order as the input data.

Extended Capabilities

Version History

Introduced in R2019b