merge r1.11 to main #4920

XuesongYang · 2022-09-12T08:20:33Z

What does this PR do ?

This PR is to merge r1.11.0 branch to the main branch. There were merge conflicts according to several updates in all domains. I fixed all conflicts. For some conflicts, I chose the main changes, while for other conflicts, I made a decision after reading detailed changes in both r1.11.0 and main.

Collection: [Note which collection this PR will affect]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

[] Make sure you read and followed Contributor guidelines
[] Did you write any new necessary tests?
[] Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Owners of PRs in every domain, please help to review the changes. Thanks.

Additional Information

Related to # (issue)

lgtm-com · 2022-09-12T08:32:21Z

This pull request introduces 4 alerts when merging 11238b0 into 8d609b3 - view on LGTM.com

new alerts:

2 for Syntax error
1 for Unused import
1 for Variable defined multiple times

redoctopus

I believe there are a few files that need to be kept from main due to the change in G2P module locations from #4690, @AlexGrinch could you take a look to check what files need to be updated?

A few that I've caught that should be kept as the version from main:

nemo/collections/tts/torch/g2ps.py
nemo/collections/tts/torch/tts_dataset.yaml
nemo/collections/tts/models/fastpitch.py
nemo/collections/tts/torch/tts_tokenizers.py
tutorials/tts/FastPitch_MixerTTS_Training.ipynb

tutorials/tts/FastPitch_MixerTTS_Training.ipynb

Signed-off-by: ericharper <complex451@gmail.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com>

…treaming. (#4687) * added support for models trained with full context. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * dropped seq_range Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed indexing in caching methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated docs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * addressed comments. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com>

* [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fix style for import Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>

* added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed style Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * included comments Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com>

Signed-off-by: ericharper <complex451@gmail.com>

Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update to reflect new config names Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Sentencepiece fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix finetuning Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add finetune eval script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix name Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update Jenkinsfile Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update check Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Backward compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com>

* g2p docs added Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix references Signed-off-by: ekmb <ebakhturina@nvidia.com> * address review feedback Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com>

* Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Updated inference code and squad scripts Signed-off-by: Virginia Adams <vadams@nvidia.com> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <vadams@nvidia.com> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added padded vocab size Signed-off-by: Virginia Adams <vadams@nvidia.com> * Fixed val check interval value Signed-off-by: Virginia Adams <vadams@nvidia.com> * Python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added back dp=1 check Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>

…c-dec models (#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Revert pin memory Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>

* Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: ericharper <complex451@gmail.com>

* Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug with DDP initialization Signed-off-by: Tim Moon <tmoon@nvidia.com> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix code formatting Signed-off-by: Tim Moon <tmoon@nvidia.com> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com>

Signed-off-by: ericharper <complex451@gmail.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

…huish diff Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

ericharper

LGTM. Thanks!

tutorials/tts/FastPitch_MixerTTS_Training.ipynb

reverted to main path.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

vadam5

LGTM

XuesongYang · 2022-09-12T23:25:43Z

Blocking this as I'm seeing a bunch of CI tests are being skipped but the CI is showing as passed.

Thanks for pointing it out. I rebased to the latest main. It would be related to the most recent PR that removed computer vision codes and tests.
#4907

README.rst

* update branch Signed-off-by: ericharper <complex451@gmail.com> * update package info and dockerfile Signed-off-by: ericharper <complex451@gmail.com> * [TTS] bugfix for missing configs. (NVIDIA#4725) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix pynini install in TTS tutorials (NVIDIA#4729) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * [TTS] updated config with a German IPA phoneme tokenizer (NVIDIA#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update r1.11 to new heteronyms list (NVIDIA#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix tutorial formatting (NVIDIA#4778) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * update branch and typos (NVIDIA#4788) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * Adding support for models trained with full context for cache-aware streaming. (NVIDIA#4687) * added support for models trained with full context. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * dropped seq_range Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed indexing in caching methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated docs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * addressed comments. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> * Update megatron encoder decoder model to support py37 for colab (NVIDIA#4791) * [ASR] Add pretrained ASR models for Croatian (NVIDIA#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fix style for import Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * added/fixed export for Megatron models (NVIDIA#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed style Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * included comments Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update branch for qa notebook Signed-off-by: ericharper <complex451@gmail.com> * Fix initializing weights from ptl ckpt with exclude (NVIDIA#4807) Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> * Fix index error from addition of voiced_mask and p_voiced (NVIDIA#4811) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * T5 prompt learning fixes (NVIDIA#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update to reflect new config names Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Sentencepiece fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix finetuning Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add finetune eval script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix name Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update Jenkinsfile Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update check Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Backward compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * G2P docs (NVIDIA#4841) * g2p docs added Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix references Signed-off-by: ekmb <ebakhturina@nvidia.com> * address review feedback Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix providing glue in seq2seq eval (NVIDIA#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Updated inference code and squad scripts (NVIDIA#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <vadams@nvidia.com> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <vadams@nvidia.com> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added padded vocab size Signed-off-by: Virginia Adams <vadams@nvidia.com> * Fixed val check interval value Signed-off-by: Virginia Adams <vadams@nvidia.com> * Python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added back dp=1 check Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Set the number of workers to 0 for validation and test sets in all enc-dec models (NVIDIA#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Revert pin memory Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (NVIDIA#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added utf8 encoding (NVIDIA#4892) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * update readme with apex commit Signed-off-by: ericharper <complex451@gmail.com> * Add support for Apex distributed Adam optimizer with GPT-3 (NVIDIA#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug with DDP initialization Signed-off-by: Tim Moon <tmoon@nvidia.com> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix code formatting Signed-off-by: Tim Moon <tmoon@nvidia.com> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * fixed styles Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed unsued import. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed duplicated func defintion. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * replace 'r1.11.0' with 'main' in Jenkinsfile and all tutorials. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix: PRE_RELEASE = 'rc0' Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * replace branch name to main for asr_with_adapters.ipynb. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix Fastpitch mixertts tutorial format to align with main to distingshuish diff Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix: correct path for tokenizers. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: ericharper <complex451@gmail.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: anteju <108555623+anteju@users.noreply.github.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>

* update branch Signed-off-by: ericharper <complex451@gmail.com> * update package info and dockerfile Signed-off-by: ericharper <complex451@gmail.com> * [TTS] bugfix for missing configs. (NVIDIA#4725) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix pynini install in TTS tutorials (NVIDIA#4729) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * [TTS] updated config with a German IPA phoneme tokenizer (NVIDIA#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update r1.11 to new heteronyms list (NVIDIA#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix tutorial formatting (NVIDIA#4778) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * update branch and typos (NVIDIA#4788) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * Adding support for models trained with full context for cache-aware streaming. (NVIDIA#4687) * added support for models trained with full context. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * dropped seq_range Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed indexing in caching methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated docs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * addressed comments. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> * Update megatron encoder decoder model to support py37 for colab (NVIDIA#4791) * [ASR] Add pretrained ASR models for Croatian (NVIDIA#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fix style for import Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * added/fixed export for Megatron models (NVIDIA#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed style Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * included comments Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update branch for qa notebook Signed-off-by: ericharper <complex451@gmail.com> * Fix initializing weights from ptl ckpt with exclude (NVIDIA#4807) Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> * Fix index error from addition of voiced_mask and p_voiced (NVIDIA#4811) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * T5 prompt learning fixes (NVIDIA#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update to reflect new config names Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Sentencepiece fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix finetuning Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add finetune eval script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix name Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update Jenkinsfile Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update check Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Backward compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * G2P docs (NVIDIA#4841) * g2p docs added Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix references Signed-off-by: ekmb <ebakhturina@nvidia.com> * address review feedback Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix providing glue in seq2seq eval (NVIDIA#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Updated inference code and squad scripts (NVIDIA#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <vadams@nvidia.com> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <vadams@nvidia.com> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added padded vocab size Signed-off-by: Virginia Adams <vadams@nvidia.com> * Fixed val check interval value Signed-off-by: Virginia Adams <vadams@nvidia.com> * Python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added back dp=1 check Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Set the number of workers to 0 for validation and test sets in all enc-dec models (NVIDIA#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Revert pin memory Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (NVIDIA#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added utf8 encoding (NVIDIA#4892) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * update readme with apex commit Signed-off-by: ericharper <complex451@gmail.com> * Add support for Apex distributed Adam optimizer with GPT-3 (NVIDIA#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug with DDP initialization Signed-off-by: Tim Moon <tmoon@nvidia.com> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix code formatting Signed-off-by: Tim Moon <tmoon@nvidia.com> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * fixed styles Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed unsued import. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed duplicated func defintion. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * replace 'r1.11.0' with 'main' in Jenkinsfile and all tutorials. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix: PRE_RELEASE = 'rc0' Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * replace branch name to main for asr_with_adapters.ipynb. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix Fastpitch mixertts tutorial format to align with main to distingshuish diff Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix: correct path for tokenizers. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: ericharper <complex451@gmail.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: anteju <108555623+anteju@users.noreply.github.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com>

XuesongYang requested review from redoctopus, ekmb and ericharper September 12, 2022 17:30

redoctopus previously requested changes Sep 12, 2022

View reviewed changes

tutorials/tts/FastPitch_MixerTTS_Training.ipynb Show resolved Hide resolved

ericharper and others added 25 commits September 12, 2022 12:39

update branch

c92c486

Signed-off-by: ericharper <complex451@gmail.com>

update package info and dockerfile

0c5e980

Signed-off-by: ericharper <complex451@gmail.com>

[TTS] bugfix for missing configs. (#4725)

8e8d822

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Fix pynini install in TTS tutorials (#4729)

5fa9e13

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Fix tutorial formatting (#4778)

c61769b

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

update branch and typos (#4788)

26f7dac

Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com>

Update megatron encoder decoder model to support py37 for colab (#4791)

fa2542e

update branch for qa notebook

04431ea

Signed-off-by: ericharper <complex451@gmail.com>

Fix initializing weights from ptl ckpt with exclude (#4807)

212ede2

Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com>

Fix index error from addition of voiced_mask and p_voiced (#4811)

d44bdd7

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

G2P docs (#4841)

ae686e1

* g2p docs added Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix references Signed-off-by: ekmb <ebakhturina@nvidia.com> * address review feedback Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com>

added utf8 encoding (#4892)

7c286ed

Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com>

update readme with apex commit

d6af5c8

Signed-off-by: ericharper <complex451@gmail.com>

update readme

1bfd9cb

Signed-off-by: ericharper <complex451@gmail.com>

fix: PRE_RELEASE = 'rc0'

120a4c3

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

ericharper mentioned this pull request Sep 12, 2022

Merge r1.11 main #4912

Closed

8 tasks

XuesongYang added 2 commits September 12, 2022 14:25

replace branch name to main for asr_with_adapters.ipynb.

ceb4157

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

fix Fastpitch mixertts tutorial format to align with main to distings…

db06e61

…huish diff Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

XuesongYang force-pushed the xueyang-merge-r1.11-to-main branch from 47b25a4 to db06e61 Compare September 12, 2022 21:51

ericharper previously approved these changes Sep 12, 2022

View reviewed changes

XuesongYang dismissed ericharper’s stale review via 7727332 September 12, 2022 22:02

XuesongYang marked this pull request as ready for review September 12, 2022 22:03

XuesongYang requested review from redoctopus and ericharper September 12, 2022 22:03

redoctopus reviewed Sep 12, 2022

View reviewed changes

tutorials/tts/FastPitch_MixerTTS_Training.ipynb Outdated Show resolved Hide resolved

XuesongYang requested a review from redoctopus September 12, 2022 22:55

XuesongYang changed the title ~~Xueyang merge r1.11 to main~~ merge r1.11 to main Sep 12, 2022

fix: correct path for tokenizers.

b679599

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

XuesongYang force-pushed the xueyang-merge-r1.11-to-main branch from bd3e2e6 to b679599 Compare September 12, 2022 22:59

vadam5 approved these changes Sep 12, 2022

View reviewed changes

XuesongYang commented Sep 12, 2022

View reviewed changes

README.rst Show resolved Hide resolved

XuesongYang requested review from okuchaiev and titu1994 September 13, 2022 00:07

ericharper merged commit cf83c5f into main Sep 13, 2022

ericharper deleted the xueyang-merge-r1.11-to-main branch September 13, 2022 00:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

merge r1.11 to main #4920

merge r1.11 to main #4920

XuesongYang commented Sep 12, 2022

lgtm-com bot commented Sep 12, 2022

redoctopus left a comment

ericharper left a comment

vadam5 left a comment

XuesongYang commented Sep 12, 2022 •

edited

Loading

merge r1.11 to main #4920

merge r1.11 to main #4920

Conversation

XuesongYang commented Sep 12, 2022

What does this PR do ?

Changelog

Usage

Before your PR is "Ready for review"

Who can review?

Additional Information

lgtm-com bot commented Sep 12, 2022

redoctopus left a comment

Choose a reason for hiding this comment

ericharper left a comment

Choose a reason for hiding this comment

vadam5 left a comment

Choose a reason for hiding this comment

XuesongYang commented Sep 12, 2022 • edited Loading

XuesongYang commented Sep 12, 2022 •

edited

Loading