Skip to content
This repository has been archived by the owner on Nov 16, 2023. It is now read-only.

add deconvnet and unet correctness train, val and test metrics as a unit test - facilitates certainty in results #318

Closed
maxkazmsft opened this issue May 26, 2020 · 0 comments
Assignees
Labels
Prior: High Type: Bug Something isn't working Type: Correctness anything to do with repo being technically correct

Comments

@maxkazmsft
Copy link
Contributor

maxkazmsft commented May 26, 2020

add performance metrics similar to test/cicd/src/check_performance.py to quantify whether the models are training correctly on train and test sets on dummy synthetic checkerboard dataset

eventually this should be added as a unit test to increase code coverage

Patch-deconvnet: Need to investigate why the metrics are not close to perfect on checkerboard dataset, like with the UNet model – Pixel Accuracy has to be close to 1 on both training and validation set, Class Accuracy has to be close to 1.0, FW IoU, IoU, etc

image

For UNet model, modifying check_performance.py is relatively straightforward - metrics come out near-perfect on checkerboard dataset. But for patch-devonvnet, debugging is needed.

@maxkazmsft maxkazmsft added Prior: High Type: Correctness anything to do with repo being technically correct Type: Bug Something isn't working labels May 26, 2020
@maxkazmsft maxkazmsft changed the title add correctness train, val and test metrics as a unit test - facilitates certainty in results add deconvnet and unet correctness train, val and test metrics as a unit test - facilitates certainty in results May 28, 2020
@maxkazmsft maxkazmsft added this to the V0.1.3 milestone Jun 3, 2020
@maxkazmsft maxkazmsft self-assigned this Jun 3, 2020
@maxkazmsft maxkazmsft mentioned this issue Jun 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Prior: High Type: Bug Something isn't working Type: Correctness anything to do with repo being technically correct
Projects
None yet
Development

No branches or pull requests

1 participant