megengine.functional.nn.layer_norm

layer_norm(inp, normalized_shape, affine, weight=None, bias=None, eps=1e-05)[source]

Applies layer normalization to the input.

Refer to LayerNorm for more information.

Parameters

  • inp (Tensor) – input tensor.

  • normalized_shape (tuple) – the shape that you want to be normalizated See normalized_shape in LayerNorm.

  • affine (bool) – whether to use learnable affine parameters (weight, bias)

  • weight (Optional[Tensor]) – scaling tensor in the learnable affine parameters. See \(\gamma\) in LayerNorm.

  • bias (Optional[Tensor]) – bias tensor in the learnable affine parameters. See \(\beta\) in LayerNorm.

  • eps (float) – a value added to the denominator for numerical stability. Default: 1e-5

previous

megengine.functional.nn.sync_batch_norm

next

megengine.functional.nn.local_response_norm