megengine.functional.nn.cross_entropy¶

cross_entropy(pred, label, axis=1, with_logits=True, label_smooth=0, reduction='mean')[源代码]¶

Computes the multi-class cross entropy loss (using logits by default).

By default(with_logitis is True), pred is assumed to be logits, class probabilities are given by softmax.

It has better numerical stability compared with sequential calls to softmax and cross_entropy.

When using label smoothing, the label distribution is as follows:

\[y^{LS}_{k}=y_{k}\left(1-\alpha\right)+\alpha/K\]

where \(y^{LS}\) and \(y\) are new label distribution and origin label distribution respectively. k is the index of label distribution. \(\alpha\) is label_smooth and \(K\) is the number of classes.

参数

pred (Tensor) – input tensor representing the predicted probability.
label (Tensor) – input tensor representing the classification label.
axis (int) – an axis along which softmax will be applied. Default: 1
with_logits (bool) – whether to apply softmax first. Default: True
label_smooth (float) – a label smoothing of parameter that can re-distribute target distribution. Default: 0
reduction (str) – the reduction to apply to the output: ‘none’ | ‘mean’ | ‘sum’. Default: ‘mean’

返回类型

Tensor

返回

loss value.

实际案例

import numpy as np
from megengine import tensor
import megengine.functional as F

data_shape = (1, 2)
label_shape = (1, )
pred = tensor(np.array([0, 0], dtype=np.float32).reshape(data_shape))
label = tensor(np.ones(label_shape, dtype=np.int32))
loss = F.nn.cross_entropy(pred, label)
print(loss.numpy().round(decimals=4))

Outputs:

0.6931

megengine.functional.nn.binary_cross_entropy

megengine.functional.nn.cvt_color