2583 Add DatasetSummary #2616

yiheng-wang-nv · 2021-07-16T09:35:42Z

Signed-off-by: Yiheng Wang vennw@nvidia.com

Fixes #2583 .

Description

For 3D image segmentation tasks, users may not sure how to set a reasonable voxel spacing value, as well as the normalization parameters, this PR implements a class to address these issues.

Status

Ready

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
Breaking change (fix or new feature that would cause existing functionality to change).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

yiheng-wang-nv · 2021-07-16T09:57:17Z

The initial implementation requires joblib, let me choose another way and then reopen this PR.

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

yiheng-wang-nv · 2021-07-16T16:03:48Z

Hi all, could you please help to review this PR?
I think this is needed, but I'm not sure if I put the class into the right place. Or should I use a tutorial rather than a MONAI feature here?

madil90 · 2021-07-19T02:24:33Z

/build

madil90 · 2021-07-19T08:00:48Z

/build

wyli · 2021-07-19T08:57:53Z

Hi all, could you please help to review this PR?
I think this is needed, but I'm not sure if I put the class into the right place. Or should I use a tutorial rather than a MONAI feature here

Hi all, could you please help to review this PR?
I think this is needed, but I'm not sure if I put the class into the right place. Or should I use a tutorial rather than a MONAI feature here?

I think this could be part of monai/data/utils.

The implemention could use pytorch or monai dataloader instead of the multiprocessing lib
it's currently hard coded to use a datalist as in put, but it should be more flexible to take monai dataset as an input
would be great to reuse this utility to generate the relevant statistics of the dataset,

MONAI/monai/handlers/utils.py

Line 200 in f943ab4

{

instead of hardcoding the type of stats to be median/min/max

monai/data/dataset_calculator.py

yiheng-wang-nv · 2021-07-20T14:36:47Z

Hi all, could you please help to review this PR?
I think this is needed, but I'm not sure if I put the class into the right place. Or should I use a tutorial rather than a MONAI feature here

Hi all, could you please help to review this PR?
I think this is needed, but I'm not sure if I put the class into the right place. Or should I use a tutorial rather than a MONAI feature here?

I think this could be part of monai/data/utils.

The implemention could use pytorch or monai dataloader instead of the multiprocessing lib

it's currently hard coded to use a datalist as in put, but it should be more flexible to take monai dataset as an input

would be great to reuse this utility to generate the relevant statistics of the dataset,

MONAI/monai/handlers/utils.py

Line 200 in f943ab4

{

instead of hardcoding the type of stats to be median/min/max

Thanks @wyli . My question is that when the dataset has images in different spatial sizes, it cannot support batch_size > 1 directly. Therefore, this PR used multiple processing instead. Do you have any suggestions?

wyli · 2021-07-20T14:39:02Z

Hi all, could you please help to review this PR?
I think this is needed, but I'm not sure if I put the class into the right place. Or should I use a tutorial rather than a MONAI feature here

Hi all, could you please help to review this PR?
I think this is needed, but I'm not sure if I put the class into the right place. Or should I use a tutorial rather than a MONAI feature here?

I think this could be part of monai/data/utils.

The implemention could use pytorch or monai dataloader instead of the multiprocessing lib

it's currently hard coded to use a datalist as in put, but it should be more flexible to take monai dataset as an input

would be great to reuse this utility to generate the relevant statistics of the dataset,

MONAI/monai/handlers/utils.py

Line 200 in f943ab4

{

instead of hardcoding the type of stats to be median/min/max

Thanks @wyli . My question is that when the dataset has images in different spatial sizes, it cannot support batch_size > 1 directly. Therefore, this PR used multiple processing instead. Do you have any suggestions?

the dataloader is implemented with a proper multiprocessing support, I think you can use it with batch_size=1

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

yiheng-wang-nv · 2021-07-28T05:46:01Z

Hi all, could you please help to review this PR?
I think this is needed, but I'm not sure if I put the class into the right place. Or should I use a tutorial rather than a MONAI feature here

Hi all, could you please help to review this PR?
I think this is needed, but I'm not sure if I put the class into the right place. Or should I use a tutorial rather than a MONAI feature here?

I think this could be part of monai/data/utils.

The implemention could use pytorch or monai dataloader instead of the multiprocessing lib

it's currently hard coded to use a datalist as in put, but it should be more flexible to take monai dataset as an input

would be great to reuse this utility to generate the relevant statistics of the dataset,

MONAI/monai/handlers/utils.py

Line 200 in f943ab4

{

instead of hardcoding the type of stats to be median/min/max

Hi @wyli , thanks for the comments. The code may not be able to be in monai/data/utils, since it needs to import dataset and dataloader and putting into monai/data/utils could lead to circular import.

Now I've switched to use monai dataloader.
The input has been changed to use monai dataset.
As for the write_metrics_reports handler, it needs to calculate with all accumulated image intensities. I think the way I updated in the latest PR may be faster (see calculate_statistics function with can get mean, std, max and min and do not need to accumulate all intensities).

Could you please help to re-review the code, thanks!

yiheng-wang-nv · 2021-07-28T05:46:58Z

Hi @ericspod , I've updated the code according to your comments, could you please help to re-review it, thanks!

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

wyli · 2021-07-28T14:17:25Z

looks like a useful utility, I'd suggest changing its name to DatasetSummary, what do you think?

ericspod · 2021-07-28T23:45:53Z

monai/data/dataset_calculator.py

+            all_intensities.append(intensities)
+
+        all_intensities = list(chain(*all_intensities))
+        self.data_min_percentile, self.data_max_percentile = np.percentile(


We're still accumulating all the foreground pixel values from all the images here, unless we have some way of doing an incremental percentile calculation we are stuck with that.

monai/data/dataset_calculator.py

ericspod

I think we're good with a few minor changes possibly.

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

yiheng-wang-nv · 2021-07-29T15:18:28Z

Hi @wyli , I've changed the name, and also add a test case for anisotropic spacings.

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

…nv/MONAI into 2583-add-dataset-calculator

wyli · 2021-07-30T08:58:06Z

Perhaps in another iteration we can change the code to use some online histogram estimation such as https://github.com/maki-nage/distogram

yiheng-wang-nv · 2021-07-30T14:10:58Z

Mark: later I will update a tutorial in https://github.com/Project-MONAI/tutorials/tree/master/modules/dynunet_pipeline to show how can we get the spacings and normalization parameters.

* Add DatasetCalculator Signed-off-by: Yiheng Wang <vennw@nvidia.com> * update docstring Signed-off-by: Yiheng Wang <vennw@nvidia.com> * use multiprocessing Signed-off-by: Yiheng Wang <vennw@nvidia.com> * update to use dataset and other places Signed-off-by: Yiheng Wang <vennw@nvidia.com> * update to support array return Signed-off-by: Yiheng Wang <vennw@nvidia.com> * update with new testcases and change name Signed-off-by: Yiheng Wang <vennw@nvidia.com> * update min test Signed-off-by: Yiheng Wang <vennw@nvidia.com> * update unittest Signed-off-by: Yiheng Wang <vennw@nvidia.com> * fix vstack error Signed-off-by: Yiheng Wang <vennw@nvidia.com>

yiheng-wang-nv added 2 commits July 16, 2021 17:15

Add DatasetCalculator

8f411d0

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

update docstring

1921f6f

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

yiheng-wang-nv closed this Jul 16, 2021

use multiprocessing

7ac7159

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

yiheng-wang-nv reopened this Jul 16, 2021

yiheng-wang-nv requested review from Nic-Ma, ericspod, rijobro and wyli July 16, 2021 15:36

ericspod reviewed Jul 19, 2021

View reviewed changes

monai/data/dataset_calculator.py Outdated Show resolved Hide resolved

ericspod reviewed Jul 19, 2021

View reviewed changes

monai/data/dataset_calculator.py Outdated Show resolved Hide resolved

ericspod reviewed Jul 19, 2021

View reviewed changes

monai/data/dataset_calculator.py Outdated Show resolved Hide resolved

ericspod reviewed Jul 19, 2021

View reviewed changes

monai/data/dataset_calculator.py Outdated Show resolved Hide resolved

update to use dataset and other places

85fda19

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

yiheng-wang-nv and others added 2 commits July 28, 2021 13:48

update to support array return

0ab5b0c

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

Merge branch 'dev' into 2583-add-dataset-calculator

a97a0c2

ericspod reviewed Jul 28, 2021

View reviewed changes

monai/data/dataset_calculator.py Outdated Show resolved Hide resolved

ericspod self-requested a review July 28, 2021 23:51

ericspod approved these changes Jul 28, 2021

View reviewed changes

yiheng-wang-nv and others added 4 commits July 29, 2021 23:04

update with new testcases and change name

4b0a875

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

update min test

20bd230

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

update unittest

e334e25

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

Merge branch 'dev' into 2583-add-dataset-calculator

d190ee0

yiheng-wang-nv added 2 commits July 30, 2021 11:47

fix vstack error

8f0369e

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

Merge branch '2583-add-dataset-calculator' of github.com:yiheng-wang-…

aedc081

…nv/MONAI into 2583-add-dataset-calculator

Merge branch 'dev' into 2583-add-dataset-calculator

c59b14e

wyli enabled auto-merge (squash) July 30, 2021 09:24

wyli changed the title ~~2583 Add DatasetCalculator~~ 2583 Add DatasetSummary Jul 30, 2021

wyli merged commit 390fe7f into Project-MONAI:dev Jul 30, 2021

yiheng-wang-nv deleted the 2583-add-dataset-calculator branch July 30, 2021 14:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

2583 Add DatasetSummary #2616

2583 Add DatasetSummary #2616

yiheng-wang-nv commented Jul 16, 2021

yiheng-wang-nv commented Jul 16, 2021

yiheng-wang-nv commented Jul 16, 2021

madil90 commented Jul 19, 2021

madil90 commented Jul 19, 2021

wyli commented Jul 19, 2021

yiheng-wang-nv commented Jul 20, 2021

wyli commented Jul 20, 2021

yiheng-wang-nv commented Jul 28, 2021

yiheng-wang-nv commented Jul 28, 2021

wyli commented Jul 28, 2021

ericspod Jul 28, 2021

ericspod left a comment

yiheng-wang-nv commented Jul 29, 2021

wyli commented Jul 30, 2021

yiheng-wang-nv commented Jul 30, 2021

2583 Add DatasetSummary #2616

2583 Add DatasetSummary #2616

Conversation

yiheng-wang-nv commented Jul 16, 2021

Description

Status

Types of changes

yiheng-wang-nv commented Jul 16, 2021

yiheng-wang-nv commented Jul 16, 2021

madil90 commented Jul 19, 2021

madil90 commented Jul 19, 2021

wyli commented Jul 19, 2021

yiheng-wang-nv commented Jul 20, 2021

wyli commented Jul 20, 2021

yiheng-wang-nv commented Jul 28, 2021

yiheng-wang-nv commented Jul 28, 2021

wyli commented Jul 28, 2021

ericspod Jul 28, 2021

Choose a reason for hiding this comment

ericspod left a comment

Choose a reason for hiding this comment

yiheng-wang-nv commented Jul 29, 2021

wyli commented Jul 30, 2021

yiheng-wang-nv commented Jul 30, 2021