Fix expert grad scaling problem with ZeRO optimizer#6546
Merged
tohtana merged 7 commits intodeepspeedai:masterfrom wyooyw:fix_expert_weight_grad_with_zeroOct 23, 2024
+119-10
Commits
Commits on Sep 17, 2024
- committed
wangyiou
Commits on Sep 18, 2024
- committedwangyiou
- committedwangyiou
