Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Extend AMP selection choices #4116

Closed
danieltudosiu opened this issue Apr 12, 2022 · 1 comment · Fixed by #4132
Closed

Extend AMP selection choices #4116

danieltudosiu opened this issue Apr 12, 2022 · 1 comment · Fixed by #4132
Assignees
@Nic-Ma
Labels
Feature request
Milestone
Bug Fixes or Misc...

Comments

@danieltudosiu
Copy link
Contributor

Is your feature request related to a problem? Please describe.
AMP creates instability during training

Describe the solution you'd like
Extend the AMP dtype selection as HuggingFace suggests here.
@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 13, 2022 •
edited
Loading

Hi @danieltudosiu ,

Thanks very much for your feature request.
Seems it's necessary to support the new A100 AMP features in the engines with torch.cuda.amp.autocast():
https://pytorch.org/docs/stable/amp.html#torch.cuda.amp.autocast.
Let me think about it and get back soon.

Thanks.

@Nic-Ma Nic-Ma self-assigned this Apr 13, 2022
@Nic-Ma Nic-Ma added this to MONAI 0.9 Apr 13, 2022
@Nic-Ma Nic-Ma added this to the Bug Fixes or Misc improvements milestone Apr 13, 2022
@Nic-Ma Nic-Ma mentioned this issue Apr 14, 2022
Merged
7 tasks
@Nic-Ma Nic-Ma moved this to Done in MONAI 0.9 Apr 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Nic-Ma Nic-Ma

Labels
Feature request
Projects
No open projects
MONAI 0.9
Status: Done
Milestone
Bug Fixes or Misc improvements
Development

Successfully merging a pull request may close this issue.

4116 Add support for advanced args of AMP Nic-Ma/MONAI
2 participants
@danieltudosiu @Nic-Ma