This repository has been archived by the owner on Jul 29, 2023. It is now read-only.
Master tests #149
Merged
Master tests #149
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #149 +/- ##
==========================================
- Coverage 80.59% 75.21% -5.38%
==========================================
Files 50 53 +3
Lines 3756 4539 +783
==========================================
+ Hits 3027 3414 +387
- Misses 729 1125 +396
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
|
@smguo @JohannaRahm I rebased this branch with respect to master. I had to make some changes to Johanna's code to make it work with 3D data and metrics orientations other than xy. Please let me know if you'd like to review again, otherwise I'll merge to master. |
|
Since nobody is jumping on this exciting opportunity to do a re-review, I'm merging. :-) |
jennyfolkesson
added a commit
that referenced
this pull request
May 31, 2022
* fixed generate meta tests * fixed inference script tests * fixed metrics tests * fixed preprocessing (masks, tile, mp aux) * fixed evaluation metrics tests * lots of issues with inference. fixed 2d so far * fixed 2.5d inference, still working on 3d * 3 remaining errors in 3d inference * debugging 3d inference * debugging 3d inference * debugging 3d inference, fixed overlap shape * fixed 5d tiling bug * fixed 3d inference! * fixed stitch tests * fixed dataset tests * fixed dataset w mask tests * fixed inference dataset tests * fixed plot utils tests * added flatfield tests * fixed gen mask tests * working on tiling tests * fixed tile nonuni tests * fixed uniform tile tests * fixed aux utils tests * debugging image utils * updated pandas version to avoid attribute error in pandas * fixed image utils tests * fixed mask utils tests * fixed mp utils tests * debugging tile utils * fixed tile utils tests * newer mpl version * updated skimage, debugging flatfield * sort output of os.listdir * initial work on moving intensity computations * added metadata check in preprocessing script * adding tests for intensity meta * simplified ints_meta_generator, added test * debugging preprocessing tests * fixed preprocess tests * added mp utils test * adding meta utils tests * adding meta tests * debugging zscore test * fixed zscore test * added preprocessing tests for meta gen and norm * f ixed plot utils test * Master tests (#149) * fixed generate meta tests * fixed inference script tests * fixed metrics tests * fixed preprocessing (masks, tile, mp aux) * fixed evaluation metrics tests * lots of issues with inference. fixed 2d so far * fixed 2.5d inference, still working on 3d * 3 remaining errors in 3d inference * debugging 3d inference * debugging 3d inference * debugging 3d inference, fixed overlap shape * fixed 5d tiling bug * fixed 3d inference! * fixed stitch tests * fixed dataset tests * fixed dataset w mask tests * fixed inference dataset tests * fixed plot utils tests * added flatfield tests * fixed gen mask tests * working on tiling tests * fixed tile nonuni tests * fixed uniform tile tests * fixed aux utils tests * debugging image utils * updated pandas version to avoid attribute error in pandas * fixed image utils tests * fixed mask utils tests * fixed mp utils tests * debugging tile utils * fixed tile utils tests * newer mpl version * updated skimage, debugging flatfield * sort output of os.listdir * Inference features (progressbar, define prediction folder name, metrics in figures) (#147) * minor fix for preprocessing_script * changed the layout of sub panels in predicted figures * removed redundant slice margin adjustment * disable the check for modelcheckpoint monitor * Added flags to save predicted images in image directory or model directory * Added data normalization options to preprocessing * updated image_inference script with data normalization * bug fix * updated tests * updated tests and 3d preprocessing * added dataset otsu mask type * added script for pooling multiple datasets * created meta_utils; added multi-processing option to meta_generator; estimate dataset z-score parameters from foreground images only * added function to sample values at block centers * added blocks_meta.csv * bug fix * updated functions to sample pixels and compute zscore parameters * Fixed the bug caused by mixed numpy datatypes; make data normalization backward compatible * updated metrics_script to new normalization options added get_pp_config function to preprocess_utils * unzscore prediction before computing SSIM; added multi-threading to tiling * unzscore predition for 3D inference * bug fix * turned off normalization for reading 3D target images for computing metrics * change output dtype to float32 * bug fix * made metrics_script backward compatible * Rename "workers" to "num_workers" in the training config * fixed tests * fixed tests * added pool config file * made Maskgenerator backward compatible * Add large 2D inference * bug fix * computed metrics stats for single FOV * update inference_script.py * bug fix * generate mask meta for user supplied masks; add watershed * update config files * update notebook * update conda env yaml * edit config * edit notebook * Add README for the notebook * edit comment * update comment blocks * update comment blocks * update README.md * update notebook * update README.md * fix plots not displayed issue * update README.md * adding a shell script for setup * shell script for setup * update setup script * update README.md * update notebook * edited the image translation exercies for clarity, added TOC, and added jupyterlab to conda environment config. * update README * update the paths to data to avoid conflict with 04_image_translation repo * update instructions * move README to course repo, clean up the notebook * fix typos * update paths for backup tiles * bux fix * update plotting * add config for 2.5D model; add warning for too small min_fraction * fix margin issue with 2.5D inference * fix indexing issues in uniform tiling * bug fix * bug fix * bug fix * bug fix * bug fix * fix plotting bug * fix tiling z indexing bug * fix inference 2.5D model bug * fix inference 2.5D model bug, cleaned * inference: add progressbar, colorbar for figure-target, decrease margin in figure * fix figure metric assignment * fix inference input middle slice selection * plot multiple inputs in figures * add tqdm to requirements.txt file * inference refactoring * changed pd version, convert nan to none when reading meta * fixed generate meta tests * fixed inference script tests * fixed metrics tests * fixed preprocessing (masks, tile, mp aux) * fixed evaluation metrics tests * lots of issues with inference. fixed 2d so far * fixed 2.5d inference, still working on 3d * 3 remaining errors in 3d inference * debugging 3d inference * debugging 3d inference * debugging 3d inference, fixed overlap shape * fixed 5d tiling bug * fixed 3d inference! * fixed stitch tests * fixed dataset tests * fixed dataset w mask tests * fixed inference dataset tests * fixed plot utils tests * added flatfield tests * fixed gen mask tests * working on tiling tests * fixed tile nonuni tests * fixed uniform tile tests * fixed aux utils tests * debugging image utils * updated pandas version to avoid attribute error in pandas * fixed image utils tests * fixed mask utils tests * fixed mp utils tests * debugging tile utils * fixed tile utils tests * newer mpl version * updated skimage, debugging flatfield * sort output of os.listdir * changed pd version, convert nan to none when reading meta * making tests compatible with progress bar changes, still debugging 3d * removed requirement to run xy metrics, fixed tests for 3d inference * fixed plot utils test * sorted glob output Co-authored-by: Johanna Rahm <48733135+JohannaRahm@users.noreply.github.com> * fixed inference script tests * fixed metrics tests * fixed preprocessing (masks, tile, mp aux) * fixed evaluation metrics tests * lots of issues with inference. fixed 2d so far * fixed 2.5d inference, still working on 3d * 3 remaining errors in 3d inference * debugging 3d inference * debugging 3d inference * debugging 3d inference, fixed overlap shape * fixed 5d tiling bug * fixed 3d inference! * working on tiling tests * fixed aux utils tests * updated skimage, debugging flatfield * sort output of os.listdir * initial work on moving intensity computations * added metadata check in preprocessing script * adding tests for intensity meta * simplified ints_meta_generator, added test * debugging preprocessing tests * fixed preprocess tests * added mp utils test * adding meta utils tests * adding meta tests * debugging zscore test * fixed zscore test * added preprocessing tests for meta gen and norm * updated docstring * switched from None to NaN due to efficiency in pandas * smguo feedback * trying to fix protocol error Co-authored-by: Johanna Rahm <48733135+JohannaRahm@users.noreply.github.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've updated tests for master. I've also updated the micro_dl codebase since the current master was broken for certain use cases like segmentation and 3D inference.
I've also updated some packages to avoid errors and upgrade soon to be obsolete functions. Updated packages are:
While I hope my changes to the codebase will not break anything for the way you're currently running microDL, the tests don't cover everything so it would be great if someone could test and see if everything still runs as expected for some current data. I look forward to your feedback. Thanks!