Replies: 2 comments 1 reply
-
|
Please see #2461 |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Then to train GPT-2 model, I have to use deep speed v0.3.0 and GPT from NVIDIA? |
Beta Was this translation helpful? Give feedback.
-
|
The general ZeRO-Offload support, excluding DPU, is available in DeepSpeed master branch. Please see this tutorial. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I wonder that is there any code which is used in experiment method of ZeRO-Offload paper.
thank you
Beta Was this translation helpful? Give feedback.
All reactions