AllreduceCallback

class AllreduceCallback(reduce_method, group=WORLD, backend=None)

具有张量融合优化的 Allreduce 回调函数。

参数:

• reduce_method (str) – the method to reduce gradiants.

• group (Group) – communication group.

• backend (Optional[str]) – override distributed backend in allreduce