-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
8185 - Refactor test #8231
8185 - Refactor test #8231
Conversation
Performance Before: 94.81s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_default_value_1_model 20.95s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_default_value_0_ 15.26s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_export_2_model 14.86s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_default_value_2_model 14.55s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_export_1_model 14.28s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_export_0_ Performance after: 1.62s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_ckpt_export_2_model 1.25s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_ckpt_export_default_2_model 0.64s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_ckpt_export_0_ 0.57s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_ckpt_export_1_model 0.57s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_ckpt_export_default_1_model 0.55s call tests/test_bundle_ckpt_export.py::TestCKPTExport::test_ckpt_export_default_0_ 0.01s setup tests/test_bundle_ckpt_export.py::TestCKPTExport::test_ckpt_export_0_
3e0fe76
to
1667eb7
Compare
Hi @ericspod, Just for context, last week, I started working in Jorge Cardoso's team at KCL, so I am still getting access to the cluster and waiting for a new laptop with a dedicated GPU. I am currently working on a laptop with integrated graphics, which is not very powerful. So, it has been hard for me to tackle the slow tests since they would run SUPER slow for me. This is why I moved to reorganise the tests. Since this obscures any further changes in these files on the tracking records, I suggest you review this part first, and I create a new pull request for the other points in your issue. Please let me know what you think. Many thanks |
Hi @garciadias I see that the tests have been moved which is good, but there's a few files that seem to have gone missing and some conflicts that need to be resolved. I'm sure you've kept what you were working on earlier so we can come back to that when you're ready. These changes are going to represent a significant change from what's in the tests folder so we'll have to discuss with other developers more. We discussed it briefly at the Core meeting but we should again, what I can suggest now is that the directory structure doesn't have to be so deep as you have it, so perhaps the Good to hear you've come on board with us, we should meet in person and discuss things further. Thanks! |
Hi @ericspod, thank you very much for reviewing this and for the welcoming regards. Missing files:When you say there are some missing files, are you referring to these file files?
If these are all missing files you are referring to, I can confirm that git marked them as deleted, but they are present at:
I don't understand why these were flagged as deleted. Conflicts:I will do my best to merge the current dev branch to this and keep solving the conflicts while we wait for approval of this change. Folder depth:Fair enough, I will move them. I am happy to keep adjusting until we are satisfied with the result. Please keep me posted on the conversation with the core group. Many thanks, Eric. |
for more information, see https://pre-commit.ci
Hi @garciadias There were 2 files mentioned as deleted in the "Files changed" section here, the second of these |
Thank you, @ericspod. I have now restored the I will be working on solving the other issues. Am I pushing too often? I see the whole pipeline is triggered. Should I accumulate some changes before pushing? |
Signed-off-by: R. Garcia-Dias <rafaelagd@gmail.com>
Signed-off-by: R. Garcia-Dias <rafaelagd@gmail.com>
Signed-off-by: R. Garcia-Dias <rafaelagd@gmail.com>
/build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[2025-02-10T16:54:55.127Z]
[2025-02-10T16:54:55.127Z] ----------------------------------------------------------------------
[2025-02-10T16:54:55.127Z] Ran 0 tests in 0.000s
[2025-02-10T16:54:55.127Z]
[2025-02-10T16:54:55.127Z] OK
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_bundle_init_bundle.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_dints_cell.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_dints_mixop.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_regunet.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_localnet.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_bundle_get_data.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_bundle_verify_metadata.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_bundle_verify_net.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_reference_resolver.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_bundle_ckpt_export.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_bundle_push_to_hf_hub.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_config_parser.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_config_item.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_bundle_download.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_bundle_workflow.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_component_locator.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_bundle_utils.py
[2025-02-10T16:54:55.127Z] monai test runner: excluding test_bundle_trt_export.py
[2025-02-10T16:54:55.127Z] []
[2025-02-10T16:54:55.127Z] time to discover tests: 0.04873630404472351s, total cases: 0.
[2025-02-10T16:54:55.383Z] coverage
[2025-02-10T16:54:55.639Z] Combined data file .coverage/.coverage.pt250-cuda126-jenkins-monai-premerge-3411-vcdbg-6rzzn.3784.XYkZCFzx
[2025-02-10T16:54:55.639Z] Combined data file .coverage/.coverage.pt250-cuda126-jenkins-monai-premerge-3411-vcdbg-6rzzn.946.XIWTFPdx
[2025-02-10T16:54:56.202Z] /usr/local/lib/python3.10/dist-packages/coverage/report_core.py:116: CoverageWarning: Couldn't parse '/home/jenkins/agent/workspace/MONAI-premerge/monai/config-3.10.py': No source for code: '/home/jenkins/agent/workspace/MONAI-premerge/monai/config-3.10.py'. (couldnt-parse)
[2025-02-10T16:54:56.202Z] coverage._warn(msg, slug="couldnt-parse")
[2025-02-10T16:54:56.202Z] /usr/local/lib/python3.10/dist-packages/coverage/report_core.py:116: CoverageWarning: Couldn't parse '/home/jenkins/agent/workspace/MONAI-premerge/monai/config.py': No source for code: '/home/jenkins/agent/workspace/MONAI-premerge/monai/config.py'. (couldnt-parse)
[2025-02-10T16:54:56.202Z] coverage._warn(msg, slug="couldnt-parse")
I would suggest create a small pr only for "test_utils" name change to previous block daily ci.
But then it would be called |
No, what I meant was to move the changes from test.utils to test.test_utils, as updated in this PR, to another PR and merge that first. I have already modified the Blossom CI based on this, and merging it as is will break the daily CI. I guess you may need some additional time to debug the refactor; based on what I pasted, it shows that 0 tests have been run. |
I have #8335 to address this now. Let's merge that, then we can update this PR. |
Sorry about that. I think I have now fixed this. |
Related to #8185. ### Description This changes the name of `tests/utils.py` to `tests/test_utils.py` to conform to the changes introduced with the daily CICD tests. This is done in #8231 but the change is being pre-merged now to get tests working again while issues are sorted out there. This also includes changes this PR made to that file, and changes anywhere else to correctly import the module. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk>
/build |
Signed-off-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com>
/build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the contribution, LGTM now!
Related to Project-MONAI#8185. ### Description This changes the name of `tests/utils.py` to `tests/test_utils.py` to conform to the changes introduced with the daily CICD tests. This is done in Project-MONAI#8231 but the change is being pre-merged now to get tests working again while issues are sorted out there. This also includes changes this PR made to that file, and changes anywhere else to correctly import the module. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk> Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Fixes Project-MONAI#8185 ### Description ## Reorganize tests I have looked at the imports in each test file and the test title to identify which files were being tested. I mirrored the file structure of MONAI on the `tests` folder and moved the files accordingly. I used some helper scripts, but the process required substantial manual intervention. When uncertain, I moved the tests to the `integration` folder since the confusion always involved many imports, and I could not find clarity from the test name. Please review the integration folder carefully, which is the one that I feel the least confident about. ``` ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. --------- Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com> Signed-off-by: R. Garcia-Dias <rafaelagd@gmail.com> Signed-off-by: Rafael Garcia-Dias <rafaelagd@gmail.com> Signed-off-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Rafael Garcia-Dias <rd24@lihe055-pc.isd.kcl.ac.uk> Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com> Co-authored-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com> Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Fixes #8185
Description
Reorganize tests
I have looked at the imports in each test file and the test title to identify which files were being tested. I mirrored the file structure of MONAI on the
tests
folder and moved the files accordingly. I used some helper scripts, but the process required substantial manual intervention. When uncertain, I moved the tests to theintegration
folder since the confusion always involved many imports, and I could not find clarity from the test name.Please review the integration folder carefully, which is the one that I feel the least confident about.