megengine.optimizer.clip_grad_norm

clip_grad_norm(tensors, max_norm, ord=2.0)[source]

Clips gradient norm of an iterable of parameters. The norm is computed over all gradients together, as if they were concatenated into a single vector. Gradients are modified in-place.

Parameters

  • tensors (Union[Tensor, Iterable[Tensor]]) – an iterable of Tensors or a single Tensor that will have gradients normalized.

  • max_norm (float) – max norm of the gradients.

  • ord (float) – type of the used p-norm. Can be 'inf' for infinity norm. Default: 2.0

Returns

Tensor of an iterable of Tensors. Total norm of the parameter gradients (viewed as a single vector).

Return type

Return type

Examples

>>> import megengine.optimizer as optim
>>> net = Net()                                                                 
>>> original_norm = optim.clip_grad_norm(net.parameters(), max_norm=1.0, ord=2)

previous

MultiStepLR

next

megengine.optimizer.clip_grad_value