6676 port generative networks vqvae #7285

marksgraham · 2023-12-04T15:05:54Z

Partially fixes #6676

Description

Implements the VQ-VAE network, including the vector quantizer block, from MONAI Generative.

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
Breaking change (fix or new feature that would cause existing functionality to change).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

Fixes model-zoo#537 ### Types of changes  - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: KumoLiu <yunl@nvidia.com>

Signed-off-by: Mark Graham <markgraham539@gmail.com>

KumoLiu

Thanks for the pr. Several minor comments inline.
Please also fix DCO error.
And seems there also has a version issue, please also fix it. Thanks!
https://github.com/Project-MONAI/MONAI/actions/runs/7088708828/job/19291787433?pr=7285#step:8:12578

monai/networks/layers/vector_quantizer.py

monai/networks/nets/vqvae.py

tests/test_vector_quantizer.py

tests/test_vqvae.py

Signed-off-by: Mark Graham <markgraham539@gmail.com>

I, Mark Graham <markgraham539@gmail.com>, hereby add my Signed-off-by to this commit: 8cf0525 Signed-off-by: Mark Graham <markgraham539@gmail.com>

Signed-off-by: Mark Graham <markgraham539@gmail.com>

Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com> Signed-off-by: Mark Graham <markgraham539@gmail.com>

Signed-off-by: Mark Graham <markgraham539@gmail.com>

…ksgraham/MONAI into 6676_port_generative_networks_vqvae]

Signed-off-by: Mark Graham <markgraham539@gmail.com>

…r networks Signed-off-by: Mark Graham <markgraham539@gmail.com>

Signed-off-by: Mark Graham <markgraham539@gmail.com>

marksgraham · 2023-12-05T13:13:07Z

Thanks @KumoLiu , I've addressed your comments. It seems you need to address one of the DCO errors, I guess from the initial commit which got included from dev. Are you able to fix that? I'll look into properly syncing gen-ai and dev so that doesn't happen in future commits

…hub.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

I, KumoLiu <yunl@nvidia.com>, hereby add my Signed-off-by to this commit: 398e41f Signed-off-by: KumoLiu <yunl@nvidia.com>

for more information, see https://pre-commit.ci

…hub.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

for more information, see https://pre-commit.ci

I, KumoLiu <yunl@nvidia.com>, hereby add my Signed-off-by to this commit: 10ae421 Signed-off-by: KumoLiu <yunl@nvidia.com> DCO Remediation Commit for YunLiu <55491388+KumoLiu@users.noreply.github.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

….com/marksgraham/MONAI into 6676_port_generative_networks_vqvae]

…hub.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

for more information, see https://pre-commit.ci

I, KumoLiu <yunl@nvidia.com>, hereby add my Signed-off-by to this commit: e830b1f Signed-off-by: KumoLiu <yunl@nvidia.com> DCO Remediation Commit for YunLiu <55491388+KumoLiu@users.noreply.github.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com> Signed-off-by: KumoLiu <yunl@nvidia.com>

…hub.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com> Signed-off-by: KumoLiu <yunl@nvidia.com>

…hub.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

docs/source/networks.rst

Nic-Ma

It overall looks good to me.

Thanks.

Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

KumoLiu · 2023-12-07T01:54:28Z

/build

Partially fixes Project-MONAI#6676 ### Description Implements the VQ-VAE network, including the vector quantizer block, from MONAI Generative. ### Types of changes  - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: KumoLiu <yunl@nvidia.com> Signed-off-by: Mark Graham <markgraham539@gmail.com> Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com> Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com> Co-authored-by: KumoLiu <yunl@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Mark Graham <markgraham539@gmail.com>

KumoLiu and others added 6 commits December 4, 2023 03:50

Commit as an other branch

e364f16

Signed-off-by: Mark Graham <markgraham539@gmail.com>

Fixes broken test case

3aba8d9

Signed-off-by: Mark Graham <markgraham539@gmail.com>

Fixes mypy errors

3590751

Signed-off-by: Mark Graham <markgraham539@gmail.com>

Fixes for change in checkpointing before torch 1.11

c5990ed

Signed-off-by: Mark Graham <markgraham539@gmail.com>

Update docs

8cf0525

marksgraham requested a review from KumoLiu December 4, 2023 15:22

KumoLiu reviewed Dec 5, 2023

View reviewed changes

KumoLiu requested review from atbenmurray, ericspod and Nic-Ma December 5, 2023 09:16

marksgraham and others added 10 commits December 5, 2023 11:42

Try to fix torch.autocast error

19dce2f

Signed-off-by: Mark Graham <markgraham539@gmail.com>

DCO Remediation Commit for Mark Graham <markgraham539@gmail.com>

6fca8d0

I, Mark Graham <markgraham539@gmail.com>, hereby add my Signed-off-by to this commit: 8cf0525 Signed-off-by: Mark Graham <markgraham539@gmail.com>

DCO

3267824

Signed-off-by: Mark Graham <markgraham539@gmail.com>

Update monai/networks/layers/vector_quantizer.py

6e2f190

Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com> Signed-off-by: Mark Graham <markgraham539@gmail.com>

Clarify input tensors follow PyTorch convention

dad8663

Signed-off-by: Mark Graham <markgraham539@gmail.com>

Merge branch '6676_port_generative_networks_vqvae]' of github.com:mar…

82cae52

…ksgraham/MONAI into 6676_port_generative_networks_vqvae]

Rename argument to in_channels in the residual unit

5d0e225

Signed-off-by: Mark Graham <markgraham539@gmail.com>

Replace num_channels argument with channels for consistency with othe…

1b682d2

…r networks Signed-off-by: Mark Graham <markgraham539@gmail.com>

Adds 3D test cases

03dbd8a

Signed-off-by: Mark Graham <markgraham539@gmail.com>

Adds 3d test cases

5f54e00

Signed-off-by: Mark Graham <markgraham539@gmail.com>

KumoLiu and others added 8 commits December 6, 2023 10:51

Merge branch 'gen-ai-dev' into 6676_port_generative_networks_vqvae]

00935b4

DCO Remediation Commit for YunLiu <55491388+KumoLiu@users.noreply.git…

398e41f

…hub.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

DCO Remediation Commit for KumoLiu <yunl@nvidia.com>

4e98dae

I, KumoLiu <yunl@nvidia.com>, hereby add my Signed-off-by to this commit: 398e41f Signed-off-by: KumoLiu <yunl@nvidia.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

c51834e

for more information, see https://pre-commit.ci

DCO Remediation Commit for YunLiu <55491388+KumoLiu@users.noreply.git…

10ae421

…hub.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

20d0cf9

for more information, see https://pre-commit.ci

Merge branch '6676_port_generative_networks_vqvae]' of https://github…

17478b0

….com/marksgraham/MONAI into 6676_port_generative_networks_vqvae]

KumoLiu and others added 6 commits December 6, 2023 11:25

DCO Remediation Commit for YunLiu <55491388+KumoLiu@users.noreply.git…

e830b1f

…hub.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

e5f644b

for more information, see https://pre-commit.ci

DCO Remediation Commit for YunLiu <55491388+KumoLiu@users.noreply.git…

d6dc104

…hub.com> I, YunLiu <55491388+KumoLiu@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 2fc0126 Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

fix flake8

a63b80d

Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

Nic-Ma reviewed Dec 6, 2023

View reviewed changes

docs/source/networks.rst Outdated Show resolved Hide resolved

Nic-Ma approved these changes Dec 6, 2023

View reviewed changes

fix doc

ef62a06

Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>

KumoLiu merged commit b3fdfdd into Project-MONAI:gen-ai-dev Dec 7, 2023
28 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

6676 port generative networks vqvae #7285

6676 port generative networks vqvae #7285

marksgraham commented Dec 4, 2023 •

edited

Loading

KumoLiu left a comment

marksgraham commented Dec 5, 2023

Nic-Ma left a comment

KumoLiu commented Dec 7, 2023

6676 port generative networks vqvae #7285

6676 port generative networks vqvae #7285

Conversation

marksgraham commented Dec 4, 2023 • edited Loading

Description

Types of changes

KumoLiu left a comment

Choose a reason for hiding this comment

marksgraham commented Dec 5, 2023

Nic-Ma left a comment

Choose a reason for hiding this comment

KumoLiu commented Dec 7, 2023

marksgraham commented Dec 4, 2023 •

edited

Loading