megengine.functional.nn.sync_batch_norm

sync_batch_norm(inp, running_mean, running_var, weight=None, bias=None, training=False, momentum=0.9, eps=1e-05, eps_mode='additive', group=WORLD)[source]

Applies synchronized batch normalization to the input.

Refer to BatchNorm2d and BatchNorm1d for more information.

Parameters

  • inp (Tensor) – input tensor.

  • running_mean (Tensor) – tensor to store running mean.

  • running_var (Tensor) – tensor to store running variance.

  • weight (Optional[Tensor]) – scaling tensor in the learnable affine parameters. See \(\gamma\) in BatchNorm2d.

  • bias (Optional[Tensor]) – bias tensor in the learnable affine parameters. See \(\beta\) in BatchNorm2d.

  • training (bool) – a boolean value to indicate whether batch norm is performed in traning mode. Default: False

  • momentum (Union[float, Tensor]) – value used for the running_mean and running_var computation. Default: 0.9

  • eps (float) – a value added to the denominator for numerical stability. Default: 1e-5

  • eps_mode – mode of calculation for eps, “max” or “additive”. Default: “additive”

  • group – communication group, caculate mean and variance between this group. Default: WORLD

Return type

Tensor

previous

megengine.functional.nn.batch_norm

next

megengine.functional.nn.layer_norm