GELU¶

class GELU(name=None)[source]¶

Applies the element-wise function:

\[\text{GELU}(x) = x\Phi(x)\]

where \(\Phi(x)\) is the Cumulative Distribution Function for Gaussian Distribution.

Examples

>>> import numpy as np
>>> data = mge.tensor(np.array([-2,-1,0,1,2,]).astype(np.float32))
>>> gelu = M.GELU()
>>> output = gelu(data)
>>> with np.printoptions(precision=4):
...     print(output.numpy())
[-0.0455 -0.1587  0.      0.8413  1.9545]

SiLU

BatchNorm1d