Fast Gradient Clipping and Ghost Clipping #656
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
added a commit
to EnayatUllah/opacus
that referenced
this pull request
Jul 18, 2024
Summary: Pull Request resolved: pytorch#656 Changes: - GradSampleModuleGC now inherits from GradSampleModule, removing and cleaning redundant code - DPOptimizerGC now inherits from DPOptimizer, removing and cleaning redundant code - Modified PrivacyEngine to work with Ghost Clipping. Usage: pass grad_sample_mode = 'ghost' and clipping = 'ghost' in the make_private function Differential Revision: D58210796
EnayatUllah
force-pushed
the
export-D58210796
branch
from
5b9d9b9 to
57d0515
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
force-pushed
the
export-D58210796
branch
from
57d0515 to
9118442
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
force-pushed
the
export-D58210796
branch
from
9118442 to
26abc93
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
force-pushed
the
export-D58210796
branch
from
26abc93 to
d049ef5
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
added a commit
to EnayatUllah/opacus
that referenced
this pull request
Jul 19, 2024
Summary: Pull Request resolved: pytorch#656 Introducing Fast Gradient Clipping and Ghost Clipping to Opacus for memory-efficient training with DP SGD. Differential Revision: D58210796
EnayatUllah
force-pushed
the
export-D58210796
branch
from
d049ef5 to
04c7c67
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
force-pushed
the
export-D58210796
branch
from
04c7c67 to
d42b90a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
added a commit
to EnayatUllah/opacus
that referenced
this pull request
Jul 19, 2024
Summary: Pull Request resolved: pytorch#656 Introducing Fast Gradient Clipping and Ghost Clipping to Opacus for memory-efficient training with DP SGD. Differential Revision: D58210796
EnayatUllah
force-pushed
the
export-D58210796
branch
from
d42b90a to
1e53f77
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
force-pushed
the
export-D58210796
branch
from
1e53f77 to
6acd3d5
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
force-pushed
the
export-D58210796
branch
from
6acd3d5 to
311f714
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
force-pushed
the
export-D58210796
branch
from
311f714 to
d0ee04d
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
added a commit
to EnayatUllah/opacus
that referenced
this pull request
Jul 20, 2024
Summary: Pull Request resolved: pytorch#656 Introducing Fast Gradient Clipping and Ghost Clipping to Opacus for memory-efficient training with DP SGD. Reviewed By: HuanyuZhang Differential Revision: D58210796
EnayatUllah
force-pushed
the
export-D58210796
branch
from
d0ee04d to
8d7039a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
force-pushed
the
export-D58210796
branch
from
8d7039a to
01b1b39
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
added a commit
to EnayatUllah/opacus
that referenced
this pull request
Jul 21, 2024
Summary: Pull Request resolved: pytorch#656 Itroducing Fast Gradient Clipping and Ghost Clipping to Opacus for memory-efficient training with DP SGD. Reviewed By: HuanyuZhang Differential Revision: D58210796
EnayatUllah
force-pushed
the
export-D58210796
branch
from
01b1b39 to
dc162b4
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
added a commit
to EnayatUllah/opacus
that referenced
this pull request
Jul 21, 2024
Summary: Pull Request resolved: pytorch#656 Itroducing Fast Gradient Clipping and Ghost Clipping to Opacus for memory-efficient training with DP SGD. Reviewed By: HuanyuZhang Differential Revision: D58210796
EnayatUllah
force-pushed
the
export-D58210796
branch
from
dc162b4 to
92274a2
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
added a commit
to EnayatUllah/opacus
that referenced
this pull request
Jul 21, 2024
Summary: Pull Request resolved: pytorch#656 Itroducing Fast Gradient Clipping and Ghost Clipping to Opacus for memory-efficient training with DP SGD. Reviewed By: HuanyuZhang Differential Revision: D58210796
EnayatUllah
force-pushed
the
export-D58210796
branch
from
92274a2 to
be99288
Compare
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
added a commit
to EnayatUllah/opacus
that referenced
this pull request
Jul 21, 2024
Summary: Pull Request resolved: pytorch#656 Itroducing Fast Gradient Clipping and Ghost Clipping to Opacus for memory-efficient training with DP SGD. Reviewed By: HuanyuZhang Differential Revision: D58210796
EnayatUllah
force-pushed
the
export-D58210796
branch
from
be99288 to
b94ef0e
Compare
Summary: Pull Request resolved: pytorch#656 Itroducing Fast Gradient Clipping and Ghost Clipping to Opacus for memory-efficient training with DP SGD. Reviewed By: HuanyuZhang Differential Revision: D58210796
|
This pull request was exported from Phabricator. Differential Revision: D58210796 |
EnayatUllah
force-pushed
the
export-D58210796
branch
from
b94ef0e to
1a00925
Compare
|
This pull request has been merged in 670fde6. |
EnayatUllah
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introducing Fast Gradient Clipping and Ghost Clipping to Opacus for memory-efficient training with DP SGD.