Main Content

Group normalization layer

A group normalization layer normalizes a mini-batch of data across grouped subsets of channels for each observation independently. To speed up training of the convolutional neural network and reduce the sensitivity to network initialization, use group normalization layers between convolutional layers and nonlinearities, such as ReLU layers.

After normalization, the layer scales the input with a learnable scale factor
*γ* and shifts by a learnable offset *β*.

creates a group normalization layer and sets the optional `layer`

= groupNormalizationLayer(`numGroups`

,`Name,Value`

)`'Epsilon'`

, Parameters and Initialization, Learn Rate and Regularization, and `Name`

properties using one or more name-value pair arguments.
You can specify multiple name-value pair arguments. Enclose each property name in
quotes.

The group normalization operation normalizes the elements
*x _{i}* of the input by first calculating the mean

$${\widehat{x}}_{i}=\frac{{x}_{i}-{\mu}_{G}}{\sqrt{{\sigma}_{G}^{2}+\epsilon}},$$

where *ϵ* is a constant that improves numerical
stability when the variance is very small. To allow for the possibility that inputs with
zero mean and unit variance are not optimal for the operations that follow group
normalization, the group normalization operation further shifts and scales the activations
using the transformation

$${y}_{i}=\gamma {\widehat{x}}_{i}+\beta ,$$

where the offset *β* and scale factor
*γ* are learnable parameters that are updated during network
training.

[1] Wu, Yuxin, and Kaiming He. “Group
Normalization.” *ArXiv:1803.08494 [Cs]*, June 11, 2018.
http://arxiv.org/abs/1803.08494.

`batchNormalizationLayer`

| `convolution2dLayer`

| `fullyConnectedLayer`

| `instanceNormalizationLayer`

| `layerNormalizationLayer`

| `reluLayer`

| `trainingOptions`

| `trainNetwork`