# instancenorm

Normalize across each channel for each observation independently

## Syntax

``Y = instancenorm(X,offset,scaleFactor)``
``Y = instancenorm(X,offset,scaleFactor,'DataFormat',FMT)``
``Y = instancenorm(___Name,Value)``

## Description

The instance normalization operation normalizes the input data across each channel for each observation independently. To improve the convergence of training the convolutional neural network and reduce the sensitivity to network hyperparameters, use instance normalization between convolution and nonlinear operations such as `relu`.

After normalization, the operation shifts the input by a learnable offset β and scales it by a learnable scale factor γ.

The `instancenorm` function applies the layer normalization operation to `dlarray` data. Using `dlarray` objects makes working with high dimensional data easier by allowing you to label the dimensions. For example, you can label which dimensions correspond to spatial, time, channel, and batch dimensions using the `"S"`, `"T"`, `"C"`, and `"B"` labels, respectively. For unspecified and other dimensions, use the `"U"` label. For `dlarray` object functions that operate over particular dimensions, you can specify the dimension labels by formatting the `dlarray` object directly, or by using the `DataFormat` option.

Note

To apply instance normalization within a `layerGraph` object or `Layer` array, use `instanceNormalizationLayer`.

example

````Y = instancenorm(X,offset,scaleFactor)` applies the instance normalization operation to the input data `X` and transforms using the specified offset and scale factor.The function normalizes over grouped subsets of the `'S'` (spatial), `'T'` (time), and `'U'` (unspecified) dimensions of `X` for each observation in the `'C'` (channel) and `'B'` (batch) dimensions, independently.For unformatted input data, use the `'DataFormat'` option.```
````Y = instancenorm(X,offset,scaleFactor,'DataFormat',FMT)` applies the instance normalization operation to the unformatted `dlarray` object `X` with format specified by `FMT` using any of the previous syntaxes. The output `Y` is an unformatted `dlarray` object with dimensions in the same order as `X`. For example, `'DataFormat','SSCB'` specifies data for 2-D image input with format `'SSCB'` (spatial, spatial, channel, batch).```
````Y = instancenorm(___Name,Value)` specifies options using one or more name-value pair arguments in addition to the input arguments in previous syntaxes. For example, `'Epsilon',3e-5` sets the variance offset to `3e-5`. ```

## Examples

collapse all

Create randomized input data with two spatial, one channel, and one observation dimension.

```width = 12; height = 12; channels = 6; numObservations = 16; X = randn(width,height,channels,numObservations); dlX = dlarray(X,'SSCB'); ```

Create the learnable parameters.

```offset = dlarray(zeros(channels,1)); scaleFactor = dlarray(ones(channels,1));```

Calculate the instance normalization.

`dlZ = instancenorm(dlX,offset,scaleFactor);`

View the size and format of the normalized data.

`size(dlZ)`
```ans = 1×4 12 12 6 16 ```
`dims(dlZ)`
```ans = 'SSCB' ```

## Input Arguments

collapse all

Input data, specified as a formatted `dlarray`, an unformatted `dlarray`, or a numeric array.

If `X` is an unformatted `dlarray` or a numeric array, then you must specify the format using the `DataFormat` option. If `X` is a numeric array, then either `scaleFactor` or `offset` must be a `dlarray` object.

`X` must have a `'C'` (channel) dimension.

Offset β, specified as a formatted `dlarray`, an unformatted `dlarray`, or a numeric array with one nonsingleton dimension with size matching the size of the `'C'` (channel) dimension of the input `X`.

If `offset` is a formatted `dlarray` object, then the nonsingleton dimension must have label `'C'` (channel).

Scale factor γ, specified as a formatted `dlarray`, an unformatted `dlarray`, or a numeric array with one nonsingleton dimension with size matching the size of the `'C'` (channel) dimension of the input `X`.

If `scaleFactor` is a formatted `dlarray` object, then the nonsingleton dimension must have label `'C'` (channel).

### Name-Value Arguments

Specify optional pairs of arguments as `Name1=Value1,...,NameN=ValueN`, where `Name` is the argument name and `Value` is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose `Name` in quotes.

Example: `'Epsilon',3e-5` sets the variance offset to `3e-5`.

Dimension order of unformatted input data, specified as a character vector or string scalar `FMT` that provides a label for each dimension of the data.

When you specify the format of a `dlarray` object, each character provides a label for each dimension of the data and must be one of the following:

• `"S"` — Spatial

• `"C"` — Channel

• `"B"` — Batch (for example, samples and observations)

• `"T"` — Time (for example, time steps of sequences)

• `"U"` — Unspecified

You can specify multiple dimensions labeled `"S"` or `"U"`. You can use the labels `"C"`, `"B"`, and `"T"` at most once.

You must specify `DataFormat` when the input data is not a formatted `dlarray`.

Data Types: `char` | `string`

Variance offset for preventing divide-by-zero errors, specified as the comma-separated pair consisting of `'Epsilon'` and a numeric scalar greater than or equal to `1e-5`.

Data Types: `single` | `double`

## Output Arguments

collapse all

Normalized data, returned as a `dlarray`. The output `Y` has the same underlying data type as the input `X`.

If the input data `X` is a formatted `dlarray`, `Y` has the same dimension format as `X`. If the input data is not a formatted `dlarray`, `Y` is an unformatted `dlarray` with the same dimension order as the input data.

## Algorithms

The instance normalization operation normalizes the elements xi of the input by first calculating the mean μI and variance σI2 over the spatial and time dimensions for each channel in each observation independently. Then, it calculates the normalized activations as

`$\stackrel{^}{{x}_{i}}=\frac{{x}_{i}-{\mu }_{I}}{\sqrt{{\sigma }_{I}^{2}+ϵ}},$`

where ϵ is a constant that improves numerical stability when the variance is very small.

To allow for the possibility that inputs with zero mean and unit variance are not optimal for the operations that follow instance normalization, the instance normalization operation further shifts and scales the activations using the transformation

`${y}_{i}=\gamma {\stackrel{^}{x}}_{i}+\beta ,$`

where the offset β and scale factor γ are learnable parameters that are updated during network training.

## Version History

Introduced in R2021a