GroupNorm¶

class GroupNorm(num_groups, num_channels, eps=1e-05, affine=True, **kwargs)[source]¶

Applies Group Normalization over a mini-batch of inputs Refer to Group Normalization

\[y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]

The mean and standard-deviation are calculated separately over the each group. \(\\gamma\) and \(\\beta\) are learnable affine transform parameters of attr:num_channels if affine is True.

Parameters

num_groups (int) – number of groups that divided from channels.
num_channels (int) – number of channels expected in input
eps – a value added to the denominator for numerical stability. Default: 1e-5
affine – this module has learnable affine parameters (weight, bias) when affine is set to be True.

Shape:

Input: \((N, C, H, W)\) (now only support NCHW format tensor)
Output: \((N, C, H, W)\) (same shape as input)

Examples

>>> import numpy as np
>>> inp = Tensor(np.arange(2 * 3 * 4 * 4).astype(np.float32).reshape(2, 3, 4, 4))
>>> m = M.GroupNorm(3, 3)
>>> out = m(inp)
>>> out.numpy().shape
(2, 3, 4, 4)

SyncBatchNorm

InstanceNorm