SiLU¶

class SiLU(name=None)[source]¶

Applies the element-wise function:

\[\text{SiLU}(x) = \frac{x}{1 + \exp(-x)}\]

Examples

>>> import numpy as np
>>> data = mge.tensor(np.array([-2,-1,0,1,2,]).astype(np.float32))
>>> silu = M.SiLU()
>>> output = silu(data)
>>> with np.printoptions(precision=6):
...     print(output.numpy())
[-0.238406 -0.268941  0.        0.731059  1.761594]

PReLU

GELU