Implement the ability to use layernorm with resnet models #7294
Assignees
Labels
Comments
|
Hi @KumoLiu Could I work on this? |
|
@aymuos15, sure, please refer to the CONTRIBUTION GUIDE. |
|
Thank you! |
KumoLiu
added a commit
that referenced
this issue
May 23, 2024
Fixes #7294 . ### Description Adds a `norm` param to ResNet ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [x] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [x] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: Peter Kaplinsky <peterkaplinsky@gmail.com> Co-authored-by: Peter Kaplinsky <peterkaplinsky@gmail.com> Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
Batch norm can be problematic with small batch sizes and certain applications due to computing running batch statistics. Layernorm does not have this problem but it is not available to use in with monai's resnet models:
Describe the solution you'd like
Implement the ability to use layer norm with resnet models
Describe alternatives you've considered
I have tried using other models which support layer norm such as densenet but it seems to be broken also. See #7293
The text was updated successfully, but these errors were encountered: