[Fix] Convert SyncBN to BN when training on DP #772
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #772 +/- ##
==========================================
+ Coverage 89.02% 89.05% +0.03%
==========================================
Files 111 111
Lines 6043 6051 +8
Branches 969 969
==========================================
+ Hits 5380 5389 +9
Misses 467 467
+ Partials 196 195 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
|
May move to |
added 2 commits
August 11, 2021 02:04
Junjun2016
reviewed
Aug 31, 2021
Junjun2016
reviewed
Aug 31, 2021
Junjun2016
reviewed
Aug 31, 2021
Junjun2016
requested changes
Aug 31, 2021
added 2 commits
August 31, 2021 21:20
Junjun2016
approved these changes
Aug 31, 2021
|
We can import from mmcv after it merged. |
|
Can import |
Junjun2016
reviewed
Sep 11, 2021
Junjun2016
reviewed
Sep 11, 2021
|
Please upgrade mmcv requirement. |
Junjun2016
reviewed
Sep 15, 2021
There was a problem hiding this comment.
https://github.com/open-mmlab/mmsegmentation/blob/master/docs/get_started.md#installation
Update mmcv dependence of master branch.
bowenroom
pushed a commit
to bowenroom/mmsegmentation
that referenced
this pull request
Feb 25, 2022
* [Fix] Convert SyncBN to BN when training on DP. * Modify SyncBN2BN. * Add SyncBN2BN unit test. * Resolve some comments. * use mmcv official revert_sync_batchnorm * Remove local syncbn2bn unit tests. * Update mmcv version. * Fix bugs of gather model tools. * Modify warnings. * Modify docker mmcv version. * Update mmcv version table.
aravind-h-v
pushed a commit
to aravind-h-v/mmsegmentation
that referenced
this pull request
Mar 27, 2023
* add accelerate to load models with smaller memory footprint * remove low_cpu_mem_usage as it is reduntant * move accelerate init weights context to modelling utils * add test to ensure results are the same when loading with accelerate * add tests to ensure ram usage gets lower when using accelerate * move accelerate logic to single snippet under modelling utils and remove it from configuration utils * format code using to pass quality check * fix imports with isor * add accelerate to test extra deps * only import accelerate if device_map is set to auto * move accelerate availability check to diffusers import utils * format code * add device map to pipeline abstraction * lint it to pass PR quality check * fix class check to use accelerate when using diffusers ModelMixin subclasses * use low_cpu_mem_usage in transformers if device_map is not available * NoModuleLayer * comment out tests * up * uP * finish * Update src/diffusers/pipelines/stable_diffusion/safety_checker.py * finish * uP * make style Co-authored-by: Pi Esposito <piero.skywalker@gmail.com>
sibozhang
pushed a commit
to sibozhang/mmsegmentation
that referenced
this pull request
Mar 22, 2024
* modify stat.py merge_docs * unify merge_docs style * fix bugs * fix bugs
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Incompatible between DP and SyncBN
run this command without setting
--launcher:The python environment will report errors about process group:
This error is caused by
SyncBN. WhenSyncBN.trainingis True,SyncBNneed to initprocess_group. However,process_groupis only valid whenDDP.The situation that SyncBN is valid:
process_groupis not None ortorch.distributed.group.WORLDis notNone;SyncBN.trainingis False orSyncBN.eval();Based on those mentioned above, we convert SyncBN to BN when training on DP.
This PR is blocked by MMCV #1253
MMCV PR #1253 may be released in MMCV 1.3.13 version, the mmcv compatibility of mmseg 0.18 will larger than 1.3.13.
The CI will pass when MMCV 1.3.13 version release