megengine.module.init

>>> import megengine.module as M
>>> m = M.Conv2d(16, 33, 3, stride=2)
>>> M.init.msra_normal_(m.weight, mode="fan_out", nonlinearity="relu")

注解

良好的初始化策略有助于你的模型在训练时更快地收敛。

Initialization

fill_

Fills the given tensor with value val.

zeros_

Fills the given tensor with scalar value 0.

ones_

Fills the given tensor with the scalar value 1.

uniform_

Fills the given tensor with random value sampled from uniform distribution \(\mathcal{U}(\text{a}, \text{b})\).

normal_

Fills the given tensor with random value sampled from normal distribution \(\mathcal{N}(\text{mean}, \text{std}^2)\).

calculate_gain

Returns a recommended gain value (see the table below) for the given nonlinearity function.

calculate_fan_in_and_fan_out

Calculates fan_in / fan_out value for given weight tensor.

calculate_correct_fan

Calculates fan_in / fan_out value for given weight tensor, depending on given mode.

xavier_uniform_

Fills tensor with random values sampled from \(\mathcal{U}(-a, a)\) where

xavier_normal_

Fills tensor with random values sampled from \(\mathcal{N}(0, \text{std}^2)\) where

msra_uniform_

Fills tensor wilth random values sampled from \(\mathcal{U}(-\text{bound}, \text{bound})\) where

msra_normal_

Fills tensor wilth random values sampled from \(\mathcal{N}(0, \text{std}^2)\) where