Sharded Accelerator 1/n: Expose clip gradients to plugins via abstract class#4639

Merged

SeanNaren merged 10 commits intomasterfrom feature/817-fairscale-1n

Nov 12, 2020

+120-50

Commits on Nov 12, 2020

Added abstract precision plugin to expose clip_gradients function, use within accelerator to clip gradients
SeanNaren
committed
Merge branch 'master' into feature/817-fairscale-1n
SeanNaren
authored
Exclude model from override, keep optimizer (needed for sharded clip gradients), add override for O2 support apex
SeanNaren
committed
Fix doc
SeanNaren
committed
Applied codereview changes
SeanNaren
committed
Refactored clip function to encapsulate tpu changes with tpu accelerator. Default to standard clip function for vanilla torch
SeanNaren
committed
Pass correct grad clip val
SeanNaren
committed
Moved var to property
SeanNaren
committed
Merge branch 'master' into feature/817-fairscale-1n
SeanNaren
authored
Apply code review suggestions
SeanNaren
committed