Merge main to asr_normalize #7084

* update to load from ckpt Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> * load ckpt peft model Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update style Signed-off-by: arendu <adithya.r@gmail.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* add model, dataset, necessary utils and tests Signed-off-by: stevehuang52 <heh@nvidia.com> * fix tarred data Signed-off-by: stevehuang52 <heh@nvidia.com> * fix typo Signed-off-by: stevehuang52 <heh@nvidia.com> * add fvad examples and update utils Signed-off-by: stevehuang52 <heh@nvidia.com> * add copyright Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor and add tests Signed-off-by: stevehuang52 <heh@nvidia.com> * update dataset Signed-off-by: stevehuang52 <heh@nvidia.com> * update test Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * fix typos Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* bug fixes Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> * fix bugs, add preparation and evaluation scripts, add readme Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> * small fixes Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add real coverage calculation, small fixes, more debug information Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add option to pass a filelist and output folder - to handle inference from multiple input files Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * added preprocessing for yago wikipedia articles - finding yago entities and their subphrases Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * yago wiki preprocessing, sampling, pseudonormalization Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * more scripts for preparation of training examples Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add some alphabet checks Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add bert on subwords, concatenate it to bert on characters Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add calculation of character_pos_to_subword_pos Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * pdb Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * tensor join bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * double hidden_size in classifier Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * pdb Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * default index value 0 instead of -1 because index cannot be negative Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * pad index value 0 instead of -1 because index cannot be negative Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * remove pdb Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bugs, add creation of tarred dataset Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add possibility to change sequence len at inference Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * change sampling of dummy candidates at inference, add candidate info file Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix import Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * update transcription now uses info Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * write path Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * 1. add tarred dataset support(untested). 2. fix bug with ban_ngrams in indexing Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * skip short_sent if no real candidates Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix import Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add braceexpand Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug in np.ones Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug in collate Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * change tensor type to long because of error in torch.gather Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix for empty spans tensor Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * same fixes in _collate_fn for tarred dataset Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug from previous commit Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * change int types to be shorter to minimize tar size Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * refactoring of datasets and inference Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * tar by 100k examples, small fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * small fixes, add analytics script Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * Add functions for dynamic programming comparison to get best path by ngrams Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * small fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fixes to support testing on SPGISpeech Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add preprocessing for userlibri Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * some refactoring Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * some refactoring Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * move some functions to utils to reuse from other project Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * move some functions to utils to reuse from other project Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * move some functions to utils to reuse from other project Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * small refactoring before pr. Add bash-scripts reproducing evaluation Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * style fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * small fixes in inference Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix - didn't move window on last symbol Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix bug - shuffle was before truncation of sorted candidates Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * refactoring, fix some bugs Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * variour fixes. Add word_indices at inference Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add candidate positions Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move data preparation and evaluation to other repo Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add infer_reproduce_paper. Refactoring Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * refactor inference using fragment indices Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add some helper functions Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix bug with parameters order Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix bugs Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * refactoring, fix bug Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add multiple variants of adjusting start/end positions Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * more fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add unit tests, other fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix CodeQl warnings Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fixes Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> * fix bugs, add preparation and evaluation scripts, add readme Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> * small fixes Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add real coverage calculation, small fixes, more debug information Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add option to pass a filelist and output folder - to handle inference from multiple input files Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * added preprocessing for yago wikipedia articles - finding yago entities and their subphrases Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * yago wiki preprocessing, sampling, pseudonormalization Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * more scripts for preparation of training examples Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add some alphabet checks Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add bert on subwords, concatenate it to bert on characters Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add calculation of character_pos_to_subword_pos Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * pdb Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * tensor join bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * double hidden_size in classifier Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * pdb Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * default index value 0 instead of -1 because index cannot be negative Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * pad index value 0 instead of -1 because index cannot be negative Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * remove pdb Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bugs, add creation of tarred dataset Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add possibility to change sequence len at inference Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * change sampling of dummy candidates at inference, add candidate info file Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix import Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * update transcription now uses info Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * write path Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * 1. add tarred dataset support(untested). 2. fix bug with ban_ngrams in indexing Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * skip short_sent if no real candidates Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix import Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add braceexpand Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug in np.ones Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug in collate Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * change tensor type to long because of error in torch.gather Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix for empty spans tensor Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * same fixes in _collate_fn for tarred dataset Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug from previous commit Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * change int types to be shorter to minimize tar size Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * refactoring of datasets and inference Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * tar by 100k examples, small fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * small fixes, add analytics script Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * Add functions for dynamic programming comparison to get best path by ngrams Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * small fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fixes to support testing on SPGISpeech Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add preprocessing for userlibri Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * some refactoring Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * some refactoring Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * move some functions to utils to reuse from other project Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * move some functions to utils to reuse from other project Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * move some functions to utils to reuse from other project Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * small refactoring before pr. Add bash-scripts reproducing evaluation Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * style fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * small fixes in inference Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * bug fix - didn't move window on last symbol Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix bug - shuffle was before truncation of sorted candidates Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * refactoring, fix some bugs Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * variour fixes. Add word_indices at inference Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add candidate positions Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move data preparation and evaluation to other repo Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add infer_reproduce_paper. Refactoring Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * refactor inference using fragment indices Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add some helper functions Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix bug with parameters order Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix bugs Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * refactoring, fix bug Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add multiple variants of adjusting start/end positions Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * more fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add unit tests, other fixes Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix CodeQl warnings Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add script for full inference pipeline, refactoring Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add tutorial Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * take example data from HuggingFace Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add docs Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix comment Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * fix bug Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * small fixes for PR Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add some more tests Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * try to fix tests adding with_downloads Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * skip tests with tokenizer download Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> --------- Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [TTS] Implement new vocoder dataset Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Redo config structure, minor fixes Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Fix alignment logging Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Fix script usage example Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Fixed epoch LR scheduling Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Support .nemo checkpoint in FP callback Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Remove align interpolator Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Remove HiFi-GAN defaults list interpolation Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Rename weighted_sample_steps to weighted_sampling_steps_per_epoch Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com>

* deb infer Signed-off-by: Evelina <ebakhturina@nvidia.com> * deb infer Signed-off-by: Evelina <ebakhturina@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * clean up Signed-off-by: Evelina <ebakhturina@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * dont do maxlen trunc for non abs pos emb Signed-off-by: Evelina <ebakhturina@nvidia.com> * dont do maxlen trunc for non abs pos emb Signed-off-by: Evelina <ebakhturina@nvidia.com> * convert for training only Signed-off-by: Evelina <ebakhturina@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add eval test, add save .nemo for sft model Signed-off-by: Evelina <ebakhturina@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * jenkins format fix Signed-off-by: Evelina <ebakhturina@nvidia.com> * update jenkins Signed-off-by: Evelina <ebakhturina@nvidia.com> * update jenkins Signed-off-by: Evelina <ebakhturina@nvidia.com> * fix jenkins Signed-off-by: Evelina <ebakhturina@nvidia.com> * remove test, ci timeout Signed-off-by: Evelina <ebakhturina@nvidia.com> * fix for m_gpt_eval.py Signed-off-by: Evelina <ebakhturina@nvidia.com> * jenkins test Signed-off-by: Evelina <ebakhturina@nvidia.com> * fix gpt_eval with sft model Signed-off-by: Evelina <ebakhturina@nvidia.com> * revert jenkins Signed-off-by: Evelina <ebakhturina@nvidia.com> * keep float conversion for model.generate() Signed-off-by: Evelina <ebakhturina@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix inference dtype Signed-off-by: Evelina <ebakhturina@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Evelina <ebakhturina@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* TDT model pull request, initial draft Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * TDT PR WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT PR WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT PR WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * TDT WIP Signed-off-by: Hainan Xu <hainanx@nvidia.com> * addressed some review comments, part1 Signed-off-by: Hainan Xu <hainanx@nvidia.com> * addressed some review comments, part1, one line fix Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add tests for comparing TDT alphas with pytorch VS kernel computation Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add tests for comparing multiblank alphas with pytorch VS kernel computation Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add tests for fixed case computation for TDT Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments for greedy-batch decoding for TDT Signed-off-by: Hainan Xu <hainanx@nvidia.com> * include config for TDT model with stateless decoders Signed-off-by: Hainan Xu <hainanx@nvidia.com> * add reference to TDT in Readme Signed-off-by: Hainan Xu <hainanx@nvidia.com> * slight modification of config file comments Signed-off-by: Hainan Xu <hainanx@nvidia.com> * addressed more comments Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * more detailed comments for tdt kernel Signed-off-by: Hainan Xu <hainanx@nvidia.com> * one line fix Signed-off-by: Hainan Xu <hainanx@nvidia.com> * fixed small bug that results in test fails for rnnt_decoding Signed-off-by: Hainan Xu <hainanx@nvidia.com> * fixed small bug that results in test fails for rnnt_decoding Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed small bug that results in test fails for rnnt_decoding Signed-off-by: Hainan Xu <hainanx@nvidia.com> * remove unused import Signed-off-by: Hainan Xu <hainanx@nvidia.com> --------- Signed-off-by: Hainan Xu <hainanx@nvidia.com> Co-authored-by: Hainan Xu <hainanx@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* fix get param * change name --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com>

* initial POC for LDDL Bert * Finish LDDL POC * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * address comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix merge head * resolving merge * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for val/test loaders * change to new LDDL class + add winding * fix logging level * fix winding * test fix * fixes to winding * add file system * add prepemption optimizations * more logging * more prints * better logging * asfsf * add barrier * removing prints * working with mb lddl loader * final changes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update requirements file with LDDL * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert adding to requirements --------- Signed-off-by: wdykas <wdykas@nvidia.com> Co-authored-by: wdykas <73254672+wdykas@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com>

Added a visual utterance-level comparison of two ASR models Signed-off-by: George <gzelenfroind@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

#6791) * Construct FP8 amax reduction group Signed-off-by: Tim Moon <tmoon@nvidia.com> * Update Megatron-core version in CI Signed-off-by: Tim Moon <tmoon@nvidia.com> --------- Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com>

* new lora test Signed-off-by: arendu <adithya.r@gmail.com> * updates Signed-off-by: arendu <adithya.r@gmail.com> * check for chat Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> * small train set Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> * precision change Signed-off-by: arendu <adithya.r@gmail.com> * fixed typo in paths Signed-off-by: arendu <adithya.r@gmail.com> * full data with limit val batches Signed-off-by: arendu <adithya.r@gmail.com> * tp2 instead of pp2 Signed-off-by: arendu <adithya.r@gmail.com> * tp2 instead of pp2 Signed-off-by: arendu <adithya.r@gmail.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Adi Renduchintala <adithya.r@gmail.com>

Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru>

* add call to p2p overlap * update Jenkins for test --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: Eric Harper <complex451@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com>

…6793) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> Co-authored-by: Markel Sanz Ausin <markelsanz14@gmail.com>

* repro for gpt eval mp mem issue Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * add print statements for memory allocation Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * adjusted hot fix that prevents softmax on the entire output embedding,now memory bottlenecked by attention softmax which needs to be solved with FA or long attention Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using compute_logprob to configure inference Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * enable compute logprob for peft Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * remove print statements Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix ci Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added docstrings Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add missing config Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * remove truncate prompt length feature Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tensor before all gather needs to be contiguous Signed-off-by: Yang Zhang <yangzhang@nvidia.com> --------- Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: tbartley94 <tbartley@nvidia.com>

Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com>

Signed-off-by: arendu <adithya.r@gmail.com>

If datasets are stored on a read-only medium, index files cannot be created into adjacent files and an alternative directory must be specified for index mapping files. This commit adds an optional `index_mapping_dir` to the constructors. Unit tests are also added. [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Update path formatting for relative paths Signed-off-by: Greg Heinrich <gheinrich@nvidia.com>

* Add kv cache support for transformer TE path Signed-off-by: Yen-Shi Wang <yenshiw@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mark get_data_parallel_group as WAR Signed-off-by: Yen-Shi Wang <yenshiw@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Initialize process group for FP8 training Signed-off-by: Tim Moon <tmoon@nvidia.com> * Update Megatron GPT eval script for non-FP8 path Signed-off-by: Yen-Shi Wang <yenshiw@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Yen-Shi Wang <yenshiw@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Yen-Shi Wang <6960565+yen-shi@users.noreply.github.com> Co-authored-by: Yen-Shi Wang <yenshiw@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com>

* initial commit Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typos Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * tweaks to padding Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * comments Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * attempt at first working version Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typos and fixed p calculation Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removing merge artifacts Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typo Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removing unnessary imports Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * if batch split succeeded no need to conv again Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * adding channel wise split Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * adding reference to pytorch issue 80020 Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * removing time chunking methods Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * accounting for the actual self._stride value Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * limiting the fix to dw_striding subsampling Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * renamed methods Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * one more accounting for the actual self._stride value Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * support for causal convs Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * option to set conv chunking size manually * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixing imports * subsampling test Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rename variable Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * imports in test Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * more runtime checks * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * a more careful test Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * bug in causal Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix in causal Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * change_conv_chunking_factor methods Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * renamed methods Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * disabling chunking by default Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * typo Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * changing default chunking to auto Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * only split if needed Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * only split if needed Signed-off-by: Dima Rekesh <bmwshop@gmail.com> --------- Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Signed-off-by: Dima Rekesh <bmwshop@gmail.com>

Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok>

* add reference to our paper Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> * add paper reference to docs Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> --------- Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru>

Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com>

* added methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added initial code. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added initial code. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added initial code. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added config files. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed bugs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated confs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated confs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated confs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated confs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * improved f.conv1d Signed-off-by: Vahid <vnoroozi@nvidia.com> * pulled from main. Signed-off-by: Vahid <vnoroozi@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * pulled from main. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added postpostnorm. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed the target continiouse bug. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added dw_striding causal. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added print for debugging. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added print for debugging. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed causal convolutions. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added _midnorm. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed transcribe. Signed-off-by: Vahid <vnoroozi@nvidia.com> * cleaned code. Signed-off-by: Vahid <vnoroozi@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * moved back configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * moved back configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated fast emit for FC models. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated fast emit for FC models. Signed-off-by: Vahid <vnoroozi@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed bug. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed bug and addressed comments. Signed-off-by: Vahid <vnoroozi@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * dropped the test. Signed-off-by: Vahid <vnoroozi@nvidia.com> --------- Signed-off-by: Vahid <vnoroozi@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* rampup bs changes Signed-off-by: dimapihtar <dpykhtar@nvidia.com> * rampup bs changes Signed-off-by: dimapihtar <dpykhtar@nvidia.com> * fixed styling Signed-off-by: dimapihtar <dpykhtar@nvidia.com> * fix bug Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: dimapihtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com>

Signed-off-by: Dounx <imdounx@gmail.com>

Signed-off-by: arendu <adithya.r@gmail.com>

* t5 lora Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * eval lora t5 Signed-off-by: arendu <adithya.r@gmail.com> * adjust differernt lora dims Signed-off-by: arendu <adithya.r@gmail.com> * minor changes Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * bugfix for state_dict Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David <amosalla@asu.edu>

* update V_NEGATIVE_NUM constant to make better use of torch.float32 range Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * adjust backpointers dtype if U_max too large Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove print statements Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove need for user to specify model_downsample_factor Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * change model.cfg.sample_rate to model.cfg.preprocessor.sample_rate Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add check to make sure that window_stride is in model.cfg.preprocessor Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * reduce memory consumption of backpointers by making them relative instead of absolute Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update librosa.get_duration() 'filename' param to 'path' Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Do not throw error if 'text' or 'pred_text' are empty and make sure CTM filepaths in the output manifest are null Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * preprocess input text by removing any duplicate spaces and converting any newlines to spaces Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Use Utterance dataclass instead of dictionaries for keeping track of token/word/segment alignments Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * refactor so can save alignments as ctm and ass format files Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * fix bugs for saving character based ASS files and for using pred_text to do alignment Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make token level .ass file use tokens with recovered capitalization Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Do not try to generate alignment files if text or pred text is empty, or if number of tokens is too large for T Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * rename output manifest file to say '_with_output_file_paths.json' Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add flag to resegment ass subtitle file to fill available text space Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix bug in resegmentation code Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix bug which skipped some utterances if batch_size more than 1 Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * reduce memory requirements by doing torch.gather on a slice of the log probs when they are needed Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * reduce memory requirements by not saving whole v_matrix Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove any extra spaces in pred_text Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused list pred_text_all_lines Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * support using hybrid Transducer-CTC models for alignment Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * fix typo - add brackets to torch.cuda.is_available() Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * make sure token case restoration will work if superscript or subscript num is in text Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove any BOM from input text Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * pick out 1st hypotheses if there is a tuple of them Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove print statement Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add detail to error message if fail to recover capitalization of tokens Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add flag use_local_attention Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * rename additional_ctm_grouping_separator -> additional_segment_grouping_separator Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update description of additional_segment_grouping_separator Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add simple docstring to get_utt_obj function Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make docstring for add_t_start_end_to_utt_obj Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update docstrings for add_t_start_end_to_utt_obj and get_batch_variables Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update README and comments in align.py Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * change 'ground truth' -> 'reference text' in documentation Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add header Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add comments to get_utt_obj function Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * move constants so they are after imports Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add file description for make_ass_files Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * get rid of Utterance object's S attribute, and correct tests so they pass now Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove some unused variables Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove unused variable model from functions saving output files Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove unused var minimum_timestamp_duration from make_ass_files functions and return utt_obj Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * move minimum_timestamp_duration param to CTMFileConfig Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove unused enumerate and unused import Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * switch reading duration from librosa to soundfile to avoid filename/path deprecation message Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Added rouge monitoring support for t5 Signed-off-by: Matvei Novikov <mattyson.so@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Matvei Novikov <mattyson.so@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

…d Flash Attention (#6666) * move to nvidia megatron repo (#6465) (#6475) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Megatron KERPLE positional embeddings (#6478) (#6480) * [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) --------- * [TTS] whitelist broken path fix. (#6412) * [TTS] whitelist broken path fix. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- * [TTS] FastPitch speaker encoder (#6417) * Add initial codes * Remove wemb * Fix import * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore aligner loss * Add ConditionalInput * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error and support pre-trained config * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename config * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change copyright and random weight test * Add initial codes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix import error * Add initial codes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataset error * Remove reference speaker embedding * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove SV encoder * Follow comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix length type * Fix append * Move error msg * Add look-up into speaker encoder * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add valueerror msg * Move lookup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error * Rebase and Fix error * Fix spk encoder * Rename n_speakers * Follow comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix n_speakers None error --------- * Sharded manifests for tarred datasets (#6395) * testing sharded manifests * compatibility * proper fixes * adding flag tot convert_to_tarred_audio_dataset * shard_manifests conf param * propagating the shard_manifests param * propagating the shard_manifests param * distributed checks * typo * typo * fixes * fixes * fixes * fixes * fixes * fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes based on PR comments and tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes to convert_to_tarred_audio_dataset.py * reversing manifest shards flag * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests * excluding manifests from webdataset url expansion * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * expand manifest paths before attempting to cache from datastore * explicit use of UTF-8 for manifest i/o --------- * Update wfst_text_normalization.rst (#6374) Add Hungarian (incoming in NeMo-text-processing) * Support Swiglu in TP PP Conversion (#6437) (#6451) * Support Swiglu in TP PP Conversion * Guard activation * Guard activation --------- * Update NeMo_TTS_Primer.ipynb (#6436) * Update NeMo_TTS_Primer.ipynb Changed a mistake in line 782. Instead of frequency band (ie. pitch) we should write frequency bin. Note that frequency bins in FFT are not related to pitch. * Update NeMo_TTS_Primer.ipynb Corrected the description of spectrogram and mel spectrogram calculations in lines 782 & 783 and added a fourth point to the description and added a reference for more mathematical details at the end of this point. --------- * add rampup batch size support for Megatron GPT (#6424) * added rampup batch size support * added tests for rampup batch size * fixed the typos * added assertions * changed assertion rules * deleted unused imports * changed tests for rampup batch size * updated rampup batch size tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed styling * rampup batch size tests changes --------- * Meagtron encoder decoder fix for empty validation outputs (#6459) (#6461) * 1. Meagtron encoder decoder fix for empty validation outputs. * 1. Debugging. --------- * Code-Switching dataset creation - upgrading to aggregate tokenizer manifest format (#6448) * added functionality to create agg tokenizer compatible manifest for CS, flag to use this mode by default * updated README with the new agg_tokenizer_manifest flag * fixed typo in scripts/speech_recognition/code_switching/README.md * changed agg_tokenizer_manifest to is_lid_manifest --------- * Added/updated new Conformer configs (#6426) (#6467) * Update script for ngram rnnt and hat beam search decoding (#6370) * add rnnt ngram beamsearch script * add return encoding embedding option * update script * add rnnt and hat ngram decoding script * add some parameters * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add return_encoder_embeddings parameter to RNNTDecodingConfig * replace return_encoder_embeddings parameter * generalization of scipt behavior * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove return_encoder_embeddings parameter * remove return_encoder_embeddings parameter * add manual encoder_embeddings calculation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix beam_width value to 8 * fix rescoring description --------- * BERT pre-training mp fork to spawn (#6442) (#6454) * change bert fork to spawn * num_workers=0 fix --------- * fix replace_bos_with_pad not found (#6443) (#6450) * reduce workers on NMT CI (#6472) (#6474) * 1. Added KERPLE positional embeddings to encoder-decoder. * 1. Added a missing file. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixing commits. * 1. Debugging. * 1. Debugging. * 1. Debugging. * 1. Debugging. --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Signed-off-by: Jim O’Regan <jaoregan@tcd.ie> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Micha Livne <michalivne@users.noreply.github.com> Co-authored-by: Cheng-Ping Hsieh <37269846+hsiehjackson@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: Jim O’Regan <jaoregan@tcd.ie> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix an invalid link in get_data.py of ljspeech (#6456) Usage of the link in line 63 leads to downloading a html file not a tsv file, so we need to change it to a raw link. Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * 1. Added external index sample. (#6462) (#6483) Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <michalivne@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Update README to add core installation (#6488) (#6489) * update README for megatron-core * fix --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix cache aware hybrid bugs (#6466) (#6484) Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix typos (#6494) (#6495) Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add disclaimer about dataset for ASR (#6496) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * fix (#6502) datastore_path_to_webdataset_url(p) if is_datastore_path(p) and is_tarred_path(p) else p NameError: name 'is_tarred_path' is not defined Co-authored-by: George <gzelenfroind@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * fix broken links r1.18.0 (#6501) (#6504) * fix broken links * fix broken links --------- Signed-off-by: Evelina <ebakhturina@nvidia.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [TTS] Create functions for TTS preprocessing without dataloader (#6317) * [TTS] Create functions for TTS preprocessing without dataloader Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Cache aware streaming nfa (#6209) * add cache aware streaming to nemo aligner Signed-off-by: Slyne Deng <slyned@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamps generator (#6499) * [BugFix] _get_batch_preds() is forced to keep logits in decoder timestamps generators Signed-off-by: Taejin Park <tango4j@gmail.com> * Ingnore keep_logits boolean in FrameASRBatchLogits Signed-off-by: Taejin Park <tango4j@gmail.com> --------- Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Jagadeesh Balam <4916480+jbalam-nv@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [TTS] Fix FastPitch energy code (#6511) Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * fix custom forward_torch_softmax (#6512) (#6517) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [TTS] fixed broken path. (#6514) (#6518) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix normalization of impulse response in ImpulsePerturbation (#6505) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add interleaved pp support (#6498) * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Switch to megatron core Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix typos (#6523) * Fix typos Signed-off-by: smajumdar <titu1994@gmail.com> * Fix typos Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * New noise_norm perturbation based on Riva work (#6445) * Initial commit for new noise_norm perturbation Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Minor fix to random seed in perturb Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updated code to reflect feedback Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updates for feedback given by code reviewers Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updates in response to PR feedback Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Added comment about ref_mic being None Signed-off-by: Daniel Egert <degert@nvidia.com> * Updated perturb to use inspect module Signed-off-by: Daniel Egert <degert@nvidia.com> --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [TTS] Add script for computing feature stats (#6508) * [TTS] Add script for computing feature stats Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add overwrite config Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add Frame-VAD model and datasets (#6441) * add model, dataset, necessary utils and tests Signed-off-by: stevehuang52 <heh@nvidia.com> * fix tarred data Signed-off-by: stevehuang52 <heh@nvidia.com> * fix typo Signed-off-by: stevehuang52 <heh@nvidia.com> * update docstring Signed-off-by: stevehuang52 <heh@nvidia.com> * update doc Signed-off-by: stevehuang52 <heh@nvidia.com> * update doc Signed-off-by: stevehuang52 <heh@nvidia.com> * update pretrained model info Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Support dynamic length batches with GPT SFT (#6510) * Support synamic length with GPT SFT Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * make branch functional Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * added back the fast emit section to the configs. (#6540) (#6542) * added back the fast emit section to the configs. * added back the fast emit section to the configs. --------- Signed-off-by: Vahid <vnoroozi@nvidia.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * removing unnessary avoid_bfloat16_autocast_context (#6481) Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * FC models in menu (#6473) * FC models in menu Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [TTS] Add tutorials for FastPitch TTS speaker adaptation with adapters (#6431) * Add tts adapter tutorial Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Update main tutorial Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add tts adapter tutorial Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Update main tutorial Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Update tutorial Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix load .nemo error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support multi-speaker fine-tune Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Use .nemo Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Follow Comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix bug Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix bug Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix bug Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add precomputed speaker emb Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix space Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove repeated argument Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * optional batch size Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix comments in notebook Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [TTS] Create initial TTS dataset feature processors (#6507) Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * fix (#6529) (#6546) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add FastConformer Hybrid ASR models for EN, ES, IT, DE, PL, HR, UA, BY (#6549) (#6553) * Added fastconfomer hybrid asr models for en, es, it, de, pl, hr, ua, by * updated ASR docs with the fastconformer hybrid checkpoints * added the fastconformer RNNT and CTC models --------- Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> Co-authored-by: Kunal Dhawan <kunaldhawan97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add scores for FastConformer models (#6557) (#6558) Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix fp16 (#6543) (#6544) Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Patch transcribe and support offline transcribe for hybrid model (#6550) (#6559) Signed-off-by: fayejf <fayejf07@gmail.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix notebook bad json (#6561) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Change Megatron Enc Dec model to use persistent_workers (#6548) (#6552) * persistent workers * fix --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Make KenLM with PC for AggregateTokenizer and merge it (#6081) * do_lowercase, rm_punctuation Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * support beam_strategy = beam Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * black Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix config and^Cunctuation capitalization Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rm math Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * update kenlm Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * black Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add opengrm Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * mv install_beamsearch_decoders Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * punctuation_to_preserve Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Only tikenizer opion Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * Black Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * DEFAULT_TOKEN_OFFSET Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * aggregate_tokenizer Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install kenlm with more than 5gram Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * install_beamsearch_decoders Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * ngram_bin_path kenlm_bin_path Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * black Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * fix greedy PC bug Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * move global params Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * fix description and perplexity Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * fix description Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * NEMO_PATH Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * nemo:23.01 Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * License Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * description Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * isinstance Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * refactor kenlm stdin Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * black Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * add cmd arg Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * use new iter_files Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * EncDecHybridRNNTCTCModel Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * punctuation Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * train_kenlm args Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * add docstrings Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add ngram_merge docs Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * ngram_prune Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rename to ngram_merge Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rename to ngram Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * add comments Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * Ngram Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * nemo_model_file Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install_opengrm_ngram Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * install opengrm Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rename to install_opengrm.sh Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rm extra import Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * train_paths Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * text_processing Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * fix ngram_bin_path Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * DECODERS_PATH Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * farcompile Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rm text processing Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * text_processing Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * AggregateTokenizer.DummyTokenizer Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * comments Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * TextProcessingConfig Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * typo Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * doc Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * types Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * nemo_model_file Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rm assert Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * import kenlm_utils Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * return None Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * Copyright Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * 2022 Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * 2023 Signed-off-by: Nikolay Karpov <karpnv@gmail.com> --------- Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> Signed-off-by: Nikolay Karpov <karpnv@gmail.com> Co-authored-by: Nikolay Karpov <nkarpov@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * fix for running on 1 GPU. Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * temp rtd fix (#6568) (#6569) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [TTS] Add script for mapping speaker names to indices (#6509) Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * whitespace (#6574) Signed-off-by: Nikolay Karpov <karpnv@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Update manifest.py for speedup (#6565) (#6573) * Update manifest.py Re-order the checks for faster processing audio filepaths that are already absolute paths * Update manifest.py --------- Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * More streaming conformer export fixes (#6567) (#6578) Signed-off-by: Greg Clark <grclark@nvidia.com> Co-authored-by: Greg Clark <grclark@nvidia.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * user selected max_seq_len should be less than model's max_seq_len (#6333) (#6386) * user selection should not break model max limit * eval max seq length --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Adi Renduchintala <108822655+arendu@users.noreply.github.com> Co-authored-by: Adi Renduchintala <108822655+arendu@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Framework for PEFT via mixins (#6391) * init commit ptuning via mixin Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updates Signed-off-by: arendu <adithya.r@gmail.com> * gpt ptuning places virtual tokens on the left only Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * encoder input modified when pre_process is true Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * optimizer group and state dict updates Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * adapter ptuning working for pp>1 Signed-off-by: arendu <adithya.r@gmail.com> * adapter defaults Signed-off-by: arendu <adithya.r@gmail.com> * adapter ptuining config defaults Signed-off-by: arendu <adithya.r@gmail.com> * training works Signed-off-by: arendu <adithya.r@gmail.com> * loading and saving adapter only params during training Signed-off-by: arendu <adithya.r@gmail.com> * added checks and comments Signed-off-by: arendu <adithya.r@gmail.com> * clean up Signed-off-by: arendu <adithya.r@gmail.com> * checks for grad is None before calling all_reduce Signed-off-by: arendu <adithya.r@gmail.com> * load adapter .nemo file working Signed-off-by: arendu <adithya.r@gmail.com> * resume training for adapters Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * peft tuning Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor Signed-off-by: arendu <adithya.r@gmail.com> * file not needed Signed-off-by: arendu <adithya.r@gmail.com> * undo prompt learning dataset changes Signed-off-by: arendu <adithya.r@gmail.com> * undo updates to gpt prompt learning model Signed-off-by: arendu <adithya.r@gmail.com> * naming updates Signed-off-by: arendu <adithya.r@gmail.com> * decoding Signed-off-by: arendu <adithya.r@gmail.com> * predict_step in gpt_sft_model Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updates Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed inference from tuning config Signed-off-by: arendu <adithya.r@gmail.com> * no test in peft training Signed-off-by: arendu <adithya.r@gmail.com> * answer only loss and correct defaults for val_loss Signed-off-by: arendu <adithya.r@gmail.com> * hybrid adapters and ptuning Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * eval working.. Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * prepending tokens for ptuning Signed-off-by: arendu <adithya.r@gmail.com> * cleaned up eval config Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * clean up Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * default prompt template Signed-off-by: arendu <adithya.r@gmail.com> * Lora added Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support synamic length with GPT SFT Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * make branch functional Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * defaults to max_pad_length=False in GPT SFT dataset Signed-off-by: arendu <adithya.r@gmail.com> * adapter parallel_adapters to support Lora Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added early stopping by default Signed-off-by: arendu <adithya.r@gmail.com> * eval script for peft and eval config. bug fixes in predict step and added out_features to t5 adapter config Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updates Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updates Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * docs Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * better defaults Signed-off-by: arendu <adithya.r@gmail.com> * updates Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> * docs Signed-off-by: arendu <adithya.r@gmail.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: Adi Renduchintala <108822655+arendu@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * cache and reuse inputs (#6422) (#6452) Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add patches for Virtual Parallel conversion (#6589) * Add patches for Virtual Parllel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Pass `.scale` instead of scaler object to core (#6551) * pass .scale instead of scaler object to core (#6545) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * scale changes for main Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Documentation for ASR-TTS models (#6594) (#6595) * Add docs about hybrid ASR-TTS models * Add docs about text-only datasets * Add docs about ASR-TTS checkpoints * Add docs about ASR-TTS configs and training * Clean up * ASR-TTS docs: add to api, fix imports * Clean up * Wrap optional import * Revert general ASR import --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [TTS] Fix aligner nan loss in fp32 (#6435) * Fix nan loss in fp32 Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Update SDP docs (#6485) (#6596) * add info about SDP e.g. processor classes in docs * add link to SDP docs in README * address code review comments and add SDP overview diagram * Fix spelling typo --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Bug/typo fixes (#6599) Signed-off-by: Igor Gitman <igitman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Manual garbage collection with an interval (#6469) (#6482) * Manual garbage collection with an interval * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * use trainer.global_step for tracking the interval of GC --------- Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Make tensor split contiguous (#6580) (#6593) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [ASR] Fix for old models in change_attention_model (#6608) * fixes Signed-off-by: sam1373 <samuelkriman@gmail.com> * done already Signed-off-by: sam1373 <samuelkriman@gmail.com> --------- Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Update manifest.py to use os.path for get_full_path (#6598) * Update manifest.py to use os.path for get_full_path Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py to get rid of pathlib Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update manifest.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Cherry pick commits in #6601 to main (#6611) * fix write Signed-off-by: fayejf <fayejf07@gmail.com> * decoding ctc Signed-off-by: fayejf <fayejf07@gmail.com> * temp set rnnt decoding return_best_hypothesis to true Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to transcribe_speech as requested Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to speech_to_text_buffered_infer_rnnt as requested Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to speech_to_text_buffered_infer_ctc as requested Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * reflect change in asr_evaluator Signed-off-by: fayejf <fayejf07@gmail.com> * reflect som and vahid comment Signed-off-by: fayejf <fayejf07@gmail.com> * remove return_best_hy=true in transcribe_speech Signed-off-by: fayejf <fayejf07@gmail.com> * no text skip Signed-off-by: fayejf <fayejf07@gmail.com> * revert partial Signed-off-by: fayejf <fayejf07@gmail.com> --------- Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Create dummy iters to satisy len checks (#6600) (#6603) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * add GPT eval mode fix for interleaved to main (#6610) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix batch size reconf for T5 FT for multi-validation (#6582) (#6588) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Not doing CastToFloat by default (#6524) (#6563) * Not doing CastToFloat by default * Added docustring * Dummy commit --------- Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Turn autocast off when precision is fp32 (#6576) * Turn autocast off when precision is fp32 (#6554) * Turn autocast off when precision is fp32 Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * address review Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * merge Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * correct auto-merge Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * correct auto-merge Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * add to GPT SFT Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * update core commit hash in readme (#6622) (#6623) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * add hat image to docs (#6619) (#6621) Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Allow indices exchange via distributed (#6618) (#6624) Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Offline and streaming inference support for hybrid model (#6570) * streaming buffered for hybrid + ctc Signed-off-by: fayejf <fayejf07@gmail.com> * change default model_stride in eval.yaml Signed-off-by: fayejf <fayejf07@gmail.com> * add fc model_stride Signed-off-by: fayejf <fayejf07@gmail.com> * small fix Signed-off-by: fayejf <fayejf07@gmail.com> * check whether model and decoding match Signed-off-by: fayejf <fayejf07@gmail.com> * small fix Signed-off-by: fayejf <fayejf07@gmail.com> * streaming buffered for hybrid + rnnt Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * fix yaml Signed-off-by: fayejf <fayejf07@gmail.com> * reflect comment wip Signed-off-by: fayejf <fayejf07@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix Signed-off-by: fayejf <fayejf07@gmail.com> * refactor and verified Signed-off-by: fayejf <fayejf07@gmail.com> * add get_full_path to buffered Signed-off-by: fayejf <fayejf07@gmail.com> * small fix Signed-off-by: fayejf <fayejf07@gmail.com> * add RNNTDecodingConfig Signed-off-by: fayejf <fayejf07@gmail.com> * model name & instruction of changing decoding Signed-off-by: fayejf <fayejf07@gmail.com> --------- Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: fayejf <36722593+fayejf@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Patch decoding for PC models (#6630) (#6631) * Patch decoding logic for PC models * Patch decoding logic for PC models --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix wer.py where 'errors' variable was not set (#6633) (#6634) Fix wer.py where 'errors' variable was not set when both reference and hypothesis are empty strings Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Restore GPT support for interleaved pipeline parallelism (#6528) (#6613) * Restore logic for data-parallel communication with pipeline parallelism in GPT * Support dynamic attention masks in GPT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Debug typos * Debug data iterator caching with interleaved pipeline parallelism Each model chunk accesses the data iterator multiple times, so we need to cache multiple samples. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update Megatron-LM commit * Distinguish between list of data iterators and data iterator that is a list * Create dummy iters to satisy len checks * Kludge while waiting for Megatron-LM update * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set transformers offline to avoid rate limiting --------- Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Eric Harper <complex451@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add FA Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix XPOS Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add warning Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix bugs Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix attention Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix comment Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix cast dtype Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Undo xpos Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * bugfix (#6636) Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Disable interctc tests (#6638) Signed-off-by: Igor Gitman <igitman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add megatron_core to requirements (#6639) (#6640) * add megatron_core to requirements * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove from jenkins (#6642) * Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <complex451@gmail.com> * remove from jenkins Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove dup Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * sft model can use this script for eval (#6637) * sft model can use this script for eval Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * please fix me Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor Signed-off-by: arendu <adithya.r@gmail.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [TTS] Fix TTS audio preprocessing bugs (#6628) Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Move black parameters to pyproject.toml (#6647) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * ASR-TTS Models: Support hybrid RNNT-CTC, improve docs. (#6620) * ASR-TTS: support hybrid RNNT-CTC models * Do not warn on optional import * Explain adding options to config * Fix import guard docs * Add docs for ConcatDataset * Add explanation for sampling parameters * Initial docs for the enhancer model * Fix use_start_end_token parameter usage --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * fix conversion and eval (#6648) * fix conversion and eval Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Confidence ensembles implementation (#6614) * Working version to train conf model + save ensemble class Signed-off-by: Igor Gitman <igitman@nvidia.com> * Working version Signed-off-by: Igor Gitman <igitman@nvidia.com> * Remove copy of transcribe_speech.py Signed-off-by: Igor Gitman <igitman@nvidia.com> * Move models parameter to config Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add explicit parameters to transcribe Signed-off-by: Igor Gitman <igitman@nvidia.com> * Small cleanups Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add temperature and integration tests Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add more tests Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add pc removal config Signed-off-by: Igor Gitman <igitman@nvidia.com> * Cleanup Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix typo Signed-off-by: Igor Gitman <igitman@nvidia.com> * Address review comments Signed-off-by: Igor Gitman <igitman@nvidia.com> --------- Signed-off-by: Igor Gitman <igitman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Patch memory used for NeMo Megatron models (#6615) * Patch memory used for NeMo Megatron models Signed-off-by: smajumdar <titu1994@gmail.com> * Cleanup the dtype of embeddings Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refactor util function for parsing precision Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refactor util function for parsing precision Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Try patch for Megatron O2 Signed-off-by: smajumdar <titu1994@gmail.com> * Refactor to incorporate megatron amp 02 state Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refactor to incorporate megatron amp 02 state Signed-off-by: smajumdar <titu1994@gmail.com> * Correct indent Signed-off-by: smajumdar <titu1994@gmail.com> * Correct utils import Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * handle artifacts when path is dir (#6658) Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * remove upgrading setuptools in reinstall.sh (#6659) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * merge lora weights into base model (#6597) * merge lora weights into base model Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * typo fix Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor update Signed-off-by: arendu <adithya.r@gmail.com> * update copyright Signed-off-by: arendu <adithya.r@gmail.com> * eval needs to know the PEFT class Signed-off-by: arendu <adithya.r@gmail.com> * add target class in training script so that we can use it in eval Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update to work for tp1 Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set restore model path Signed-off-by: arendu <adithya.r@gmail.com> * peft can be none Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated merge script so that eval works easily Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * eval with peft or sft model Signed-off-by: arendu <adithya.r@gmail.com> * keep sentences in jsonl format Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * convert sft using correct classpath Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated to force sft yaml to have the correct target Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated docs Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix conversion and eval Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * upgrade to 23.04 (#6660) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Merge r1.18.0 bugfixes and doc updates to main (#6655) * update branch Signed-off-by: ericharper <complex451@gmail.com> * Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <complex451@gmail.com> * remove from jenkins Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> * remove dup Signed-off-by: ericharper <complex451@gmail.com> * update branch Signed-off-by: ericharper <complex451@gmail.com> * [TTS] reformat NeMo versions in the tts logging messages to avoid batch process them when upgrading NeMo versions. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> --------- Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Confidence ensembles: fix issues and add tuning functionality (#6657) * Implement compute confidence to properly handle blanks Signed-off-by: Igor Gitman <igitman@nvidia.com> * Implement proper confidence for transducers Signed-off-by: Igor Gitman <igitman@nvidia.com> * Implement tuning logic Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add tests for confidence tuning Signed-off-by: Igor Gitman <igitman@nvidia.com> * Remove unused imports Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add types/docs Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add comment about the main conf compute loop Signed-off-by: Igor Gitman <igitman@nvidia.com> --------- Signed-off-by: Igor Gitman <igitman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [TTS] Implement new TextToSpeech dataset (#6575) * [TTS] Implement new TextToSpeech dataset Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add unit tests Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Fix defaulting of use_log_energy Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Fix TTS export test Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Dialogue dataset (#6654) * chatbot interface Signed-off-by: Yi Dong <yidong@nvidia.com> * latest gradio Signed-off-by: Yi Dong <yidong@nvidia.com> * default greedy Signed-off-by: Yi Dong <yidong@nvidia.com> * better chatbot Signed-off-by: Yi Dong <yidong@nvidia.com> * handle preamble Signed-off-by: Yi Dong <yidong@nvidia.com> * added chatbot training capablity Signed-off-by: Yi Dong <yidong@nvidia.com> * added chatbot ui Signed-off-by: Yi Dong <yidong@nvidia.com> * remove debug code Signed-off-by: Yi Dong <yidong@nvidia.com> * default human Signed-off-by: Yi Dong <yidong@nvidia.com> * use special token for roles Signed-off-by: Yi Dong <yidong@nvidia.com> * special tokens Signed-off-by: Yi Dong <yidong@nvidia.com> * fix name Signed-off-by: Yi Dong <yidong@nvidia.com> * new chat dataset Signed-off-by: Yi Dong <yidong@nvidia.com> * fix the system token Signed-off-by: Yi Dong <yidong@nvidia.com> * upgrade gradio Signed-off-by: Yi Dong <yidong@nvidia.com> * save the chat history Signed-off-by: Yi Dong <yidong@nvidia.com> * update ui Signed-off-by: root <you@example.com> * update chat interface Signed-off-by: Yi Dong <yidong@nvidia.com> * handles canonical form Signed-off-by: Yi Dong <yidong@nvidia.com> * new sft chatbot Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change format Signed-off-by: Yi Dong <yidong@nvidia.com> * check extra_id in the tokenizer Signed-off-by: Yi Dong <yidong@nvidia.com> * added vocab property check Signed-off-by: Yi Dong <yidong@nvidia.com> * added missing file Signed-off-by: Yi Dong <yidong@nvidia.com> --------- Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: root <you@example.com> Co-authored-by: root <you@example.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add support for RNNT/hybrid models to partial transcribe (#6609) * Add support for RNNT/hybrid models to partial transcribe Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update transcribe_utils.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update transcribe_speech.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update transcr…

Signed-off-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Yi Dong <43824965+yidong72@users.noreply.github.com>

* add model, dataset, necessary utils and tests Signed-off-by: stevehuang52 <heh@nvidia.com> * fix tarred data Signed-off-by: stevehuang52 <heh@nvidia.com> * fix typo Signed-off-by: stevehuang52 <heh@nvidia.com> * add fvad examples and update utils Signed-off-by: stevehuang52 <heh@nvidia.com> * add copyright Signed-off-by: stevehuang52 <heh@nvidia.com> * add frame-vad to ASR+VAD pipeline, add drop-frame mode Signed-off-by: stevehuang52 <heh@nvidia.com> * fix typo Signed-off-by: stevehuang52 <heh@nvidia.com> * update doc Signed-off-by: stevehuang52 <heh@nvidia.com> * fix masking Signed-off-by: stevehuang52 <heh@nvidia.com> * update doc Signed-off-by: stevehuang52 <heh@nvidia.com> * slight refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * fix rnnt output Signed-off-by: stevehuang52 <heh@nvidia.com> * add support for hybrid model Signed-off-by: stevehuang52 <heh@nvidia.com> * update tutorial Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * fix typo Signed-off-by: stevehuang52 <heh@nvidia.com> * merge frame- and segment-vad scripts Signed-off-by: stevehuang52 <heh@nvidia.com> * update tutorial Signed-off-by: stevehuang52 <heh@nvidia.com> * update doc Signed-off-by: stevehuang52 <heh@nvidia.com> * update doc Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>

* add model pretraining and customization classes * fix * test width * increase middle pane width * add modules and datasets * remove global in t5 dataset s and fix formatting in megatron base model --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com>

fix ctc decoding for hybrid model in partial transcribe Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com>

* import Enum Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * make web server import conditional Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com>

* add trainer.validate example * clean up white space * add mbs and gbs to the config --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com>

* typo fix Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Fix k2 build topo helper: reassign modified labels attribute Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

* update container Signed-off-by: ericharper <complex451@gmail.com> * run import tests in parallel, isntall Cython Signed-off-by: ericharper <complex451@gmail.com> * fix typo Signed-off-by: ericharper <complex451@gmail.com> * remove redundant comment Signed-off-by: ericharper <complex451@gmail.com> * fix more typos Signed-off-by: ericharper <complex451@gmail.com> * upload and download wheel Signed-off-by: ericharper <complex451@gmail.com> * fix typos Signed-off-by: ericharper <complex451@gmail.com> * fix typos Signed-off-by: ericharper <complex451@gmail.com> * test order Signed-off-by: ericharper <complex451@gmail.com> * remove name Signed-off-by: ericharper <complex451@gmail.com> * fix indent Signed-off-by: ericharper <complex451@gmail.com> * add names back Signed-off-by: ericharper <complex451@gmail.com> * don't upload or download just build in parallel Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com>

Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com>

Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com>

* removed some tests Signed-off-by: arendu <adithya.r@gmail.com> * updated Signed-off-by: arendu <adithya.r@gmail.com> --------- Signed-off-by: arendu <adithya.r@gmail.com>

…6899) * Fix transcribe_utils.py for hybrid models in partial transcribe mode Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update transcribe_utils.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> --------- Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>

* hot fix SDE Signed-off-by: George <gzelenfroind@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: George <gzelenfroind@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Vitaly Lavrukhin <vlavrukhin@nvidia.com>

* fix for lora bug and makes ptuning w peft framework compatible with FT inference Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update Signed-off-by: arendu <adithya.r@gmail.com> * simple forward call for adapters with residual Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updates Signed-off-by: arendu <adithya.r@gmail.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add hybrid model support to transcribe_speech_parallel.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update audio_to_text_dataset.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> --------- Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* make gradio optinoal Signed-off-by: Yi Dong <yidong@nvidia.com> * update readme Signed-off-by: Yi Dong <yidong@nvidia.com> --------- Signed-off-by: Yi Dong <yidong@nvidia.com>

* update fvad doc Signed-off-by: stevehuang52 <heh@nvidia.com> * fix typo Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>

Signed-off-by: fayejf <36722593+fayejf@users.noreply.github.com>

* Fix fast-swiglu Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * change to all fast glu activation Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu>

* add Megatron ONNX export guide Signed-off-by: Asfiya Baig <asfiyab@nvidia.com> * fix formatting Signed-off-by: Asfiya Baig <asfiyab@nvidia.com> * include megatron_onnx_export in api.rst Signed-off-by: Asfiya Baig <asfiyab@nvidia.com> * include megatron_onnx_export in index.rst Signed-off-by: Asfiya Baig <asfiyab@nvidia.com> * update installation section Signed-off-by: Asfiya Baig <asfiyab@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert changes to megatron_ckpt_to_nemo.py Signed-off-by: Asfiya Baig <asfiyab@nvidia.com> * address comments Signed-off-by: Asfiya Baig <asfiyab@nvidia.com> --------- Signed-off-by: Asfiya Baig <asfiyab@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com>

* Fix for isolated filesystems in multi-node setting Signed-off-by: Greg Heinrich <gheinrich@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Greg Heinrich <gheinrich@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>

Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com>

* update fvad doc Signed-off-by: stevehuang52 <heh@nvidia.com> * fix typo Signed-off-by: stevehuang52 <heh@nvidia.com> * update fvad example Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>

* Set default apply_query_key_layer_scaling to false Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add cross attention test Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* oom wip Signed-off-by: arendu <adithya.r@gmail.com> * minor Signed-off-by: arendu <adithya.r@gmail.com> * comments Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* added assertion Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * added assertion Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Initial commit of online code switched dataset Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updated tests for new argument added to BPE classes in audio_to_text.py Signed-off-by: Daniel Egert <degert@nvidia.com> * Updated logic to catch bad audios with all zeros Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Added method docstrings and convert lang_probs to be an optional dict Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updated lang_probs docstring to correct type Signed-off-by: Daniel Egert <degert@nvidia.com> * Added final batch of requested changes and docs Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Made changes for final release candidate test Signed-off-by: Daniel Egert <degert@nvidia.com> * Fixed random tabs and changed some docstrings Signed-off-by: Daniel Egert <degert@nvidia.com> * Changed input types slightly Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * import guard soundfile due to CI failure in test-nlp-imports Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed one last issue with dataloader null checks Signed-off-by: Daniel Egert <degert@nvidia.com> --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Signed-off-by: trias702 <25867060+trias702@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

* Apply garbage collection inverval to validation steps * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com>

* enable rpe in bert model Signed-off-by: sararb <sara.rabhi@gmail.com> * expose position_embedding_type to config Signed-off-by: sararb <sara.rabhi@gmail.com> --------- Signed-off-by: sararb <sara.rabhi@gmail.com>

* Fix an4 dataset link in docs Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove broken a4 dataset links from tutorials Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

* Update core commit for CI Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * add comment Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* Tensor-parallel communication overlap with userbuffer backend (#6780) * add interfaces for tp_communication overlap [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Interface to provide custom userbuffer communicator settings by yaml file [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Construct MPI process group for userbuffers support Signed-off-by: Tim Moon <tmoon@nvidia.com> --------- Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * codeql change Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: ericharper <complex451@gmail.com>

* inference test Signed-off-by: arendu <adithya.r@gmail.com> * Update typo Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com>

* text gen condition on value Signed-off-by: Yi Dong <yidong@nvidia.com> * fix round function Signed-off-by: Yi Dong <yidong@nvidia.com> * predict value Signed-off-by: Yi Dong <yidong@nvidia.com> * scale 9 Signed-off-by: Yi Dong <yidong@nvidia.com> * handle hard code label Signed-off-by: Yi Dong <yidong@nvidia.com> * use likert scale 7 Signed-off-by: Yi Dong <yidong@nvidia.com> * scale 6 Signed-off-by: Yi Dong <yidong@nvidia.com> * merge the latest main Signed-off-by: Yi Dong <yidong@nvidia.com> * added latest chatbot ui Signed-off-by: Yi Dong <yidong@nvidia.com> * added new playground interface Signed-off-by: Yi Dong <yidong@nvidia.com> * default scale 9 Signed-off-by: Yi Dong <yidong@nvidia.com> * address comments Signed-off-by: Yi Dong <yidong@nvidia.com> * add speicial tokens Signed-off-by: Yi Dong <yidong@nvidia.com> * handles more tokenizer Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added comments Signed-off-by: Yi Dong <yidong@nvidia.com> * fix type Signed-off-by: Yi Dong <yidong@nvidia.com> * faster check Signed-off-by: Yi Dong <yidong@nvidia.com> --------- Signed-off-by: Yi Dong <yidong@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

…ank scenario (#6937) * fixed rnnt logprob selection logic for exclude_blank scenario Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix all blank ctc edge case Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> --------- Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add ub communicator initialization to validation step (#6807) * fix code qol Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ericharper <complex451@gmail.com>

Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com>

Signed-off-by: vnoroozi <vnoroozi@nvidia.com>

* Cut branch r1.19.0 Signed-off-by: smajumdar <titu1994@gmail.com> * Tutorial fixes (#6717) Signed-off-by: smajumdar <titu1994@gmail.com> * fix notebook error (#6840) Signed-off-by: Yi Dong <yidong@nvidia.com> * fix (#6842) Signed-off-by: Yi Dong <yidong@nvidia.com> * update branch Signed-off-by: ericharper <complex451@gmail.com> * update package info Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Yi Dong <43824965+yidong72@users.noreply.github.com>

Signed-off-by: ericharper <complex451@gmail.com>

* Fix requirements for pydantic + inflect Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update distopt API for coalesced NCCL calls Signed-off-by: Tim Moon <tmoon@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update comment Signed-off-by: Tim Moon <tmoon@nvidia.com> --------- Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Remove `compute_on_step` from metrics * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove confusing log message * Update tests --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Signed-off-by: Sergii Dymchenko <sdym@fb.com>

Signed-off-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Yi Dong <43824965+yidong72@users.noreply.github.com>

* add support for max_total_length=4096 for 43b Signed-off-by: Zhilin Wang <wangzhilin12061996@hotmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

#6989) Signed-off-by: Vadim Kantorov <vadimkantorov@gmail.com>

* Cache handling without input tensors mutation * Cleanup * Cleanup#2 * Cleanup#3 --------- Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>

* Implemented generic kv-pair setting of export_config from args * Hybrid conformer export * Hybrid decoder export * Cleanup * Changed from **kwargs * Docstring * Docs added * Stringify args * Added docs for ASR export configs * lowercase ctc --------- Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com>

* Bug fix for the confidence ensembles * Relax constraints for the test --------- Signed-off-by: Igor Gitman <igitman@nvidia.com> Co-authored-by: Igor Gitman <igitman@nvidia.com>

* [TTS] Add cosine distance option to TTS aligner Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Update aligner comments Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com>

* Initial commit of minor MPT-7B fixes Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* change timeout Signed-off-by: ericharper <complex451@gmail.com> * change to 8 hours Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com>

* remove hard coded input and output fields Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Push changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * add continue training script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [WIP] nonlinear interp Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * override encoder_seq_len Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove nonlinear Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * sft with pi (#7006) * sft with pi Signed-off-by: Evelina <ebakhturina@nvidia.com> * update values only if not None" Signed-off-by: Evelina <ebakhturina@nvidia.com> --------- Signed-off-by: Evelina <ebakhturina@nvidia.com> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add info Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Empty Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> --------- Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Evelina <ebakhturina@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Adding the confidence ensembles tutorial. Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix issues with notebook in colab Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add clarification about the last cell Signed-off-by: Igor Gitman <igitman@nvidia.com> * Move SDP installation on top of the tutorial Signed-off-by: Igor Gitman <igitman@nvidia.com> --------- Signed-off-by: Igor Gitman <igitman@nvidia.com>

* Force working space memory to always be in fp32 Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for fp16 testing in Numba Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for fp16 testing in Numba Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for fp16 testing in Numba Signed-off-by: smajumdar <titu1994@gmail.com> * Fix cost calculation by upcasting to fp32 Signed-off-by: smajumdar <titu1994@gmail.com> * Fix cost calculation by upcasting to fp32 Signed-off-by: smajumdar <titu1994@gmail.com> * Add support to check if numba fp16 is available Signed-off-by: smajumdar <titu1994@gmail.com> * add RNN-T loss implemented by PyTorch and test code (#5312) * Fix the bugs in cache-aware streaming Conformer (#5032) Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * IA3 support for GPT and T5 (#4909) * init commit for ia3 adater training in GPT Signed-off-by: arendu <adithya.r@gmail.com> * ia3 adater training in GPT, models and adapter classes Signed-off-by: arendu <adithya.r@gmail.com> * reshape to operate even on non-contiguous tensors Signed-off-by: arendu <adithya.r@gmail.com> * configs Signed-off-by: arendu <adithya.r@gmail.com> * fixed none init Signed-off-by: arendu <adithya.r@gmail.com> * adding adapter and ia3 support for T5 based models Signed-off-by: arendu <adithya.r@gmail.com> * style fix Signed-off-by: arendu <adithya.r@gmail.com> * config update and t5 model adapter and ia3 Signed-off-by: arendu <adithya.r@gmail.com> * removed unused imports Signed-off-by: arendu <adithya.r@gmail.com> * predict step for inference Signed-off-by: arendu <adithya.r@gmail.com> * style fix Signed-off-by: arendu <adithya.r@gmail.com> * style fix Signed-off-by: arendu <adithya.r@gmail.com> * adapter inference for t5 Signed-off-by: arendu <adithya.r@gmail.com> * style fix Signed-off-by: arendu <adithya.r@gmail.com> * fixed bug micro and global batch size in eval Signed-off-by: arendu <adithya.r@gmail.com> * minor edit Signed-off-by: arendu <adithya.r@gmail.com> * agressive truncation if in test examples if no truncation field is given Signed-off-by: arendu <adithya.r@gmail.com> * corrected for language_model_path name changes in main Signed-off-by: arendu <adithya.r@gmail.com> * removed unused import Signed-off-by: arendu <adithya.r@gmail.com> * name change for language_model_path Signed-off-by: arendu <adithya.r@gmail.com> * include inter_attention to IA3 Signed-off-by: arendu <adithya.r@gmail.com> * minor fix in confg Signed-off-by: arendu <adithya.r@gmail.com> * minor fixes Signed-off-by: arendu <adithya.r@gmail.com> * removed unused flag Signed-off-by: arendu <adithya.r@gmail.com> * addressing PR comments Signed-off-by: arendu <adithya.r@gmail.com> * address PR comments Signed-off-by: arendu <adithya.r@gmail.com> * minor fix Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: arendu <adithya.r@gmail.com> * CI test Signed-off-by: arendu <adithya.r@gmail.com> * minor fix in jenkinsfile Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Bug fix - Limit val batches set to 1.0 (#5023) * Bug fix Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adressed sandeep's comments * Fixing limit val batches support in bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixing limit val batches support in bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [bug_fix] kv_channels is used when available (#5066) * fix bug s.t kv_channels is used when available Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * P&C Docs (#5068) (#5069) Signed-off-by: Matvei Novikov <mattyson.so@gmail.com> Signed-off-by: Matvei Novikov <mattyson.so@gmail.com> Signed-off-by: Matvei Novikov <mattyson.so@gmail.com> Co-authored-by: Matvei Novikov <mattyson.so@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Add spe_split_by_unicode_script arg (#5072) * Add spe_split_by_unicode_script arg Signed-off-by: Anas <aabouallaban@pm.me> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Anas <aabouallaban@pm.me> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * probabilites -> probabilities (#5078) (#5079) Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * increase PR and Issue sweep quantity and active close PRs. (#5073) * increase PR and Issue sweep quantity and active close PRs. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * update with stricter rules, 30 days to be stale and 7 days to be closed for both Issues and PRs. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [TTS] added missing German phoneme tokenizer. (#5070) (#5074) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * rename to match prompt leanring (#5076) Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Missing fixes from r1.11.0 to T5 finetuning eval (#5054) (#5061) * Fixes to seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Notebook bug fixes (#5084) (#5085) * Notebook bug fixes Signed-off-by: Virginia Adams <vadams@nvidia.com> * Turned nemo install back on Signed-off-by: Virginia Adams <vadams@nvidia.com> * reverted notebook Signed-off-by: Virginia Adams <vadams@nvidia.com> * Updated one line in entity linking nb Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * update strategy in notebook from ddp_fork to dp (#5088) (#5089) Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Fix bug in Squeezeformer Conv block (#5011) (#5024) * Fix bug in Squeezeformer Conv block Signed-off-by: smajumdar <smajumdar@nvidia.com> * Fix kernel context Signed-off-by: smajumdar <smajumdar@nvidia.com> * Fix access mixin Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * fixed megatron lm conversion bug (PTL related) (#5038) (#5063) Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Fix Unhashable type list for Numba Cuda spec augment kernel (#5093) (#5094) Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Fix numba (#5098) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Make it possible to specify output_filename in normalize_with_audio.py (#5092) Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Greedy decoding confidence for CTC and RNNT (#4931) * rnnt confidence draft Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * word confidence Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * advanced entropies added Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * refactoring Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * oops forgot a file Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * metrics and benchmarking script added Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * style fix Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * texterrors installation added Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * lgtm and bug fix Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * fix comments Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * fix typos Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * add missing import after rebase Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: Aleksandr Laptev <alaptev@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [Add] SLURP models and examples (#4668) * add model, util and loss Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor annd update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update and refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * update and refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * update and refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update docs Signed-off-by: stevehuang52 <heh@nvidia.com> * update available models Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor data processing Signed-off-by: stevehuang52 <heh@nvidia.com> * fix typo Signed-off-by: stevehuang52 <heh@nvidia.com> * update docs Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor and update Signed-off-by: stevehuang52 <heh@nvidia.com> * update doc Signed-off-by: stevehuang52 <heh@nvidia.com> * move transformer to asr.modules Signed-off-by: stevehuang52 <heh@nvidia.com> * move transformer to asr.modules Signed-off-by: stevehuang52 <heh@nvidia.com> * get rid of jsonlines Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * revert changes to nlp Signed-off-by: stevehuang52 <heh@nvidia.com> Signed-off-by: stevehuang52 <heh@nvidia.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Jagadeesh Balam <4916480+jbalam-nv@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * only optimize params that are part of the adapter modules (#5086) Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Pipeline Parallel T5 Prompt Learning (#4956) * Added pre process flag checks and pipeline parallel in fwd Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added rank check for pipeline parallel Signed-off-by: Virginia Adams <vadams@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * T5 prompt learning works! Signed-off-by: Virginia Adams <vadams@nvidia.com> * IA3 passing CI Signed-off-by: Virginia Adams <vadams@nvidia.com> * Fixed typo Signed-off-by: Virginia Adams <vadams@nvidia.com> * removed optimizer setup so Adi's change will not conflict Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Adi Renduchintala <108822655+arendu@users.noreply.github.com> Co-authored-by: Adi Renduchintala <108822655+arendu@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [TTS] remove phonemizer.py (#5090) remove phonemizer.py and convert code block to markdown in the tutorial. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * T5 Decoding with PP > 2 fix (#5091) (#5103) * set sequence lenghts in the pipeline properly Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [TTS] fixed wrong val loss for epoch 0 and inconsistent metrics names (#5087) (#5102) * fixed hifigan configs as well * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Fix and refactor consumed samples save/restore for Megatron models. (#5077) * Fixes and refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove unused imports Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Empty Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * RIR corpus generator tool (#4927) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Multiprocessing fix (#5106) (#5107) Signed-off-by: Matvei Novikov <mattyson.so@gmail.com> Signed-off-by: Matvei Novikov <mattyson.so@gmail.com> Signed-off-by: Matvei Novikov <mattyson.so@gmail.com> Co-authored-by: Matvei Novikov <mattyson.so@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [Bug fix] PC lexical + audio (#5109) (#5110) * training running Signed-off-by: ekmb <ebakhturina@nvidia.com> * revert Signed-off-by: ekmb <ebakhturina@nvidia.com> * revert Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [Fix] schedulers with no max_steps param (#4564) * fix schedulers Signed-off-by: stevehuang52 <heh@nvidia.com> * update to use python inspect module Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> Signed-off-by: stevehuang52 <heh@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * T5 prompt learning fixes missing from r.11.0 merge (#5075) (#5101) * Fix special tokens Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Empty Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: David <amosalla@asu.edu> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [TTS] Add NeMo TTS Primer Tutorial (#4933) * [TTS] Add NeMo TTS Primer Tutorial Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Add Squeezeformer CTC model checkpoints on Librispeech (#5121) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * adding loss normalization options to rnnt joint (#4829) * adding normalization options to rnnt joint loss * moving the param to joint * moving loss normalization to rnnt loss config * style * cleaning up * fixing sum reduction in joint Signed-off-by: Dima Rekesh <drekesh@nvidia.com> * moving reduction into RNNT loss class * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactoring * typos Signed-off-by: Dima Rekesh <drekesh@nvidia.com> Signed-off-by: Dima Rekesh <drekesh@nvidia.com> Co-authored-by: Dima Rekesh <drekesh@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Asr concat dataloader (#5108) * forced precision * typo * initial commit Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typos and bugs Signed-off-by: Dima Rekesh <drekesh@nvidia.com> * reverting conformer encoder Signed-off-by: Dima Rekesh <drekesh@nvidia.com> * additional checks Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * adding support to CTC models as well * reverting conformer_encoder Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typo Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactoring Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactoring Signed-off-by: Dima Rekesh <drekesh@nvidia.com> * merging Signed-off-by: Dima Rekesh <drekesh@nvidia.com> Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Signed-off-by: Dima Rekesh <drekesh@nvidia.com> Co-authored-by: Dima Rekesh <drekesh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * fix blossom ci unittests Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * bugfix: pybtex.database.InvalidNameString: Too many commas in author field. (#5112) (#5115) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Uppdate container version to 22.09 (#5105) * update container version Signed-off-by: ericharper <complex451@gmail.com> * pin click Signed-off-by: ericharper <complex451@gmail.com> * pin click 8.0.2 Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Remove unsupported arguments from MegatronNMT (#5065) * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * More fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * pp2 support for T5 IA3 learning and T5 Adapters learning (#5116) * enabling pp2 Signed-off-by: arendu <adithya.r@gmail.com> * optimizer update Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * T5 pp>1 support for adapters and ia3 Signed-off-by: arendu <adithya.r@gmail.com> * fix bug with missing adapter_tuning Signed-off-by: arendu <adithya.r@gmail.com> * inference error fixed, pp=2 Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * T5 Prompt Learning Fixes for Pipeline Parallel (#5120) * Initial fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Added back validation acc Signed-off-by: Virginia Adams <vadams@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Put num workers back Signed-off-by: Virginia Adams <vadams@nvidia.com> * added relative encoding if statament Signed-off-by: Virginia Adams <vadams@selene-login-01.nvidia.com> * Added back val loss only validation Signed-off-by: Virginia Adams <vadams@nvidia.com> * Revert "Added back val loss only validation" This reverts commit 86d8f4806fe30335c40c3716ce18259939df500f. * Removed val acc for PP > 1 Signed-off-by: Virginia Adams <vadams@nvidia.com> * Removed enc_seq_len if statement Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added back validation acc calc Signed-off-by: Virginia Adams <vadams@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@selene-login-01.nvidia.com> Co-authored-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Virginia Adams <vadams@selene-login-01.nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * add doc info (#4721) Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [TTS] Add SpanishCharsTokenizer (#5135) * [TTS] Add SpanishCharsTokenizer Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Update megatron interface to dialogue (#4936) * fix style formatting Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update template to include description of intent Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * changes based on requests in review Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add compatibility with assistant dataset Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkins Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove dialogue_state_tracking Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update huggingface utils for dialogue Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * fix style Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile for SGDGEN Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile for SGDGEN Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile for SGDGEN Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile for SGDGEN Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile for SGDGEN Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * fix typo Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add docstrings for assistant data processsor Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkins for SGDGEN local checkpoint Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update style Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * use local vocab file for Jenkinsfile Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * patch for Jenkins CI using local file Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add slot filling prediction and metrics Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove unused code Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * refactor metrics code out of Dialogue GPT Model Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * integrate backward compatible support for IntentSlotClassificationModel (bert model) Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * save prediction file for IntentSlotClassification Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update dialogue gpt model training for megatron gpt Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove batch generate for HF GPT2, which causes lower performance Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add few shot capability to dialogue gpt model Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile and remove unused import Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update code description and clarity Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * address PR comments Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * integrate compatibility with ZeroShotIntentModel Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * rename folder to dialogue due to increased scope and further refactor for clarity Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * added dialogue GPT for sequence generation task (e.g. answer extender) Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add CI test for DialogueGPTGenerationModel Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * integrate DialogueS2SGenerationModel for generation task (e.g. answer extender) Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * modify huggingface utils to support HF t5/BART models Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove unused imports Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update bleu metric Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * fix bleu metric style Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * debug bleu metric Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * debug bleu metric Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update based on PR #3893 Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update 2 based on PR #3893 Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update 3 based on PR #3893 Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * integrate sgd generation based on user user utterance and system slot-values to generate system utterance Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add validation model saving capabilities Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * cleaned up code for SGD Based Answer extender Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Dialogue Generation CI Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkinsfile Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * fix Jenkins CI issue" Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add support for design dataset Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove unnecessary imports Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkins Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update jenkins Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update jenkins Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * support megatron for dialogue_s2s_generation_model Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update CI Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update checkpoint and predictions filename to include epoch number Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * integrate HF BART MNLI into zero shot intent model Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * integrate Dialogue Nearest Neighbour Model Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkins Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Jenkins Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * refactor Dialogue SGD Data Processor to make interface for models cleaner Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update jenkins Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update Dialogue S2S Generation model for DialogueSGDDataProcessor interface Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update jenkins Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update jenkins Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * support sgd and drive thru datasets by zero shot model and nearest neighbour model Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add prediction saving code to nearest neighbour and zero shot intent models Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * fix typo in sgd data processor Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * integrate Dialogue Mellon QA Data Processor Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update mellon qa Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update dialogue.py to remove outdated info Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update dialogue_config.yaml Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update dialogue_config.yaml Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add dialogue docs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * address review comments Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix for cfg Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * make dependency on apex optional Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * change NLPDDPluggin calling logic to make it possible to run without apex Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add first draft of tutorial Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * reduce ms marco size by removing lines without wellFormedAnswers Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * address pr comments Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update colab tutorial link in dialogue docs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * include unit test and some refactor to facilitate unit test Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * address pr issues Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove typos in dialogue tutorial Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * support larger files for question answering Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove unnecessary artifacts to reduce memory use Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * put 0 tensor to device Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update link within dialogue tutorial Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * restore previously delete files Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error handling when loss = nan Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update nan handling Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update spanning loss func Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update spanning loss Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * fix type error raised in qa_dataset.py Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add error checking message Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * revert back to float32 Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * revert back to float32 Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error msgs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error msgs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error msgs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error msgs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error msgs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error msgs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error msgs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error msgs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update exp logging Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error msgs Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update loading of large file from pickle to json Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update loading of large file from pickle to json Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * limit number of negative samples Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * revert post processing Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * revert post processing Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove unused methods and style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add more documentation Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove unused imports Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * changes base on PR review Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * set wandb logger falseby default Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update interface with megatron gpt prompt learning Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update inline documentation Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update prompt_ids Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update error msg Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update config Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update config Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * set inference = False for dialgue prompt learning during trainng Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * set inference = False for dialgue prompt learning during trainng Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove unused code Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update config yaml Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * fix bug for megatron gpt prompt learning Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove unused import Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * address comments in PR Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * address comments in PR Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * address typo Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * add megatron t5 inference Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * fix bug due to bert tokenizer not being space-aware Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update style Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update IntentSlotModel onnx export test Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update style Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update exportable Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * address PR comments Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * replace functools.cache_property with functools.lru_cache to maintain python 3.7 compatibility Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * improve speed of rank_candidates and support for p tuning Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update dialogue.py Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * fix megatron prompt learning saving bug Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update generate_candidate method Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * remove repeated init text ids and invert attention masks Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update typo Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * custom collate fn to remove excess padding in batch Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * style fix Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update complete method to mitigate issue when max seq len is low Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * address pr comments Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> * update generation interface Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> Signed-off-by: Zhilin Wang <zhilinw@nvidia.com> Co-authored-by: Zhilin Wang <zhilinw@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Added save inference ready .nemo file with every checkpoint (#5055) * Added save inference ready .nemo file with every checkpoint Signed-off-by: Virginia Adams <vadams@nvidia.com> * Python style fix Signed-off-by: Virginia Adams <vadams@nvidia.com> * addressed Adi's comment Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added ptuning check in model checkpoint saving Signed-off-by: Virginia Adams <vadams@nvidia.com> * Changed save_nemo_on_valdaition default to False Signed-off-by: Virginia Adams <vadams@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Changes global batch size of adapter CI Signed-off-by: Virginia Adams <vadams@nvidia.com> * Changed num workers to 0 Signed-off-by: Virginia Adams <vadams@nvidia.com> * added first stage of pipeline check Signed-off-by: Virginia Adams <vadams@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Fixes for docs/typos + remove max_utts parameter from tarred datasets as it causes hang in training (#5118) * Remove ; from jupyter notebook cells Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix typos in documentation/code Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix output message to have 'or equal' Signed-off-by: Igor Gitman <igitman@nvidia.com> * Link formatting fixes Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add error if max_utts is used in tarred datasets Signed-off-by: Igor Gitman <igitman@nvidia.com> * Remove max_utts parameter from tarred datasets Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix max_utts removal in tests Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix typo if -> is Signed-off-by: Igor Gitman <igitman@nvidia.com> Signed-off-by: Igor Gitman <igitman@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Merge r1.12.0 main (#5139) * update branch Signed-off-by: ericharper <complex451@gmail.com> * Add cherry-pick action (#4958) * add cherry-pick action Signed-off-by: ericharper <complex451@gmail.com> * Pin Transformers version to fix CI (#4955) * Pin transformers version in CI to prevent offline tokenizer loading error Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Drop version Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Disable offline temporarily Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Disable offline temporarily Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Enable offline Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> * upper bound transformers Signed-off-by: ericharper <complex451@gmail.com> * remove duplicate transformers requirement Signed-off-by: ericharper <complex451@gmail.com> * Release SOTA Lang ID model (#5080) * add pretrained lang id model ambernet Signed-off-by: fayejf <fayejf07@gmail.com> * update doc and style fix Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: fayejf <fayejf07@gmail.com> * update branch and package info Signed-off-by: ericharper <complex451@gmail.com> * remove upper bounds on lightning and transformers Signed-off-by: ericharper <complex451@gmail.com> * remove transformers offline from ci Signed-off-by: ericharper <complex451@gmail.com> * upper bound transformers Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: fayejf <fayejf07@gmail.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Added ASR model comparison to SDE (#5043) SDE: Added ASR model comparison tool to SDE transcribe speech: Added support for many predictions in one file, as well as custom field names Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * fix nmt eval sampler (#5154) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Fix Global init steps (#5143) * move global step to base Signed-off-by: Yi Dong <yidong@nvidia.com> * fix fused softmax Signed-off-by: Yi Dong <yidong@nvidia.com> * add the missing file Signed-off-by: Yi Dong <yidong@nvidia.com> * update the fused kernel Signed-off-by: Yi Dong <doyend@gmail.com> * fix import error Signed-off-by: Yi Dong <doyend@gmail.com> * fix import again Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Yi Dong <doyend@gmail.com> Co-authored-by: Yi Dong <doyend@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * [TTS] bug fix - sample rate was being ignored in vocoder dataset (#4518) * bug fix - sample rate was being ignored in vocoder dataset when not loading mel * handled n segments for a different sampling rate than original sampling rate * Added case for n_segments 0, warning for n_segments greater than file length Signed-off-by: Paarth Neekhara <paarth.n@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Add EMA support to NeMo (#4764) * Added Base files Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Some refactors, swap to using MNIST Lnet Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add a few more tests, allow the callback to be set via the exp manager Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Actually run validation for testing Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Run isort Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add test for saving state/fix saving state Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Use dummy model Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix test Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add copyright Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Support saving separate EMA weight module Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add standalone functionality/logging Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Expose more parameters Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Modify to allow option to replace validation Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add jenkins test, formatting Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Pin Transformers version to fix CI (#4955) * Pin transformers version in CI to prevent offline tokenizer loading error Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Drop version Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Disable offline temporarily Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Disable offline temporarily Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Enable offline Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add cherry-pick action (#4958) (#4961) * add cherry-pick action Signed-off-by: ericharper <complex451@gmail.com> * Pin Transformers version to fix CI (#4955) * Pin transformers version in CI to prevent offline tokenizer loading error Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Drop version Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Disable offline temporarily Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Disable offline temporarily Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Enable offline Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix changelog builder (#4962) (#4963) Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * fix cherry pick workflow (#4964) (#4965) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * reorder model check (#4959) (#4967) Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * check for active conda environment (#4970) (#4971) Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * [TTS] fix broken tutorial for MixerTTS. (#4949) (#4976) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Checkpoint averaging class fix (#4946) * 1. Added args.class_path to provide it externally. Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> * 1. Fixed style. Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add ability to give seperate datasets for test, train and validation (#4798) * Add ability to give seperate datasets for test, train and validation * Addressed Sandeeps comments * Addressed Sandeeps comments * Add ability to give seperate datasets for test, train and validation * Add ability to give seperate datasets for test, train and validation * Addressed review comments * Bug fix for common dataset utils * Add CI tests Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com> * Reformat code Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com> * Bug fix Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com> * Bug fix * Bug Fix * Bug Fix * Update Jenkinsfile * Addressed comments * Addressed Eriks comments. * Addressed Sandeep * Update Jenkinsfile * Update Jenkinsfile * Update dataset_utils.py * Update Jenkinsfile * Update Jenkinsfile * Use GPT CI config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * fix label models restoring issue from wrighted cross entropy (#4968) (#4975) Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add simple pre-commit file (#4983) * Add simple pre-commit file Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Exclude docs folder Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks" This reverts commit 053bd5ba579537a5f311b431871c21f3381b43eb. Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Import pycuda.autoprimaryctx or pycuda.autoinit to init pycuda execution environment (#4951) Signed-off-by: Jin Li <liji@nvidia.com> Signed-off-by: Jin Li <liji@nvidia.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Adding speaker embedding conditioning in fastpitch (#4986) Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com> Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix ASR issues (#4984) (#4991) * Fix ASR issues Signed-off-by: smajumdar <smajumdar@nvidia.com> * Revert fix Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix current tests Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * More test coverage Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Address reviews Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Drop bf16 test Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Address review Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * remove print Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add bf16 Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Jin Li <liji@nvidia.com> Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Micha Livne <michalivne@users.noreply.github.com> Co-authored-by: shanmugamr1992 <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: liji-nv <59594262+liji-nv@users.noreply.github.com> Co-authored-by: Subhankar Ghosh <subhankar2321@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Fix BF16 test (#5162) Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Fix errors in speaker diarization nemo docs (#5153) * fix docs and docstrings for MSDD Signed-off-by: Taejin Park <tango4j@gmail.com> * fix nemo docs errors Signed-off-by: Taejin Park <tango4j@gmail.com> * reflected review comments Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * Add interleaved pipeline schedule to GPT (#5025) * add virtual pipeline size to config Signed-off-by: ericharper <complex451@gmail.com> * convert model to list of modules Signed-off-by: ericharper <complex451@gmail.com> * convert model to list of modules Signed-off-by: ericharper <complex451@gmail.com> * convert model to list of modules Signed-off-by: ericharper <complex451@gmail.com> * update for list of modules Signed-off-by: ericharper <complex451@gmail.com> * add virtual to init Signed-off-by: ericharper <complex451@gmail.com> * update first last stage embedding all reduce Signed-off-by: ericharper <complex451@gmail.com> * update sequence parallel all reduce for virtual models Signed-off-by: ericharper <complex451@gmail.com> * runs but we get an error Signed-off-by: ericharper <complex451@gmail.com> * set virtual rank 0 after looping Signed-off-by: ericharper <complex451@gmail.com> * account for virtual when determinining first and last pipeline stages Signed-off-by: ericharper <complex451@gmail.com> * checkpointing for virtual models in progress Signed-off-by: ericharper <complex451@gmail.com> * add checkpoint hooks Signed-off-by: ericharper <complex451@gmail.com> * working on validation when resuming Signed-off-by: ericharper <complex451@gmail.com> * skip sanity val steps by default in config Signed-off-by: ericharper <complex451@gmail.com> * remove comment Signed-off-by: ericharper <complex451@gmail.com> * log number of params Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style Signed-off-by: ericharper <complex451@gmail.com> * check if self.model is a list Signed-off-by: ericharper <complex451@gmail.com> * make virtual pipeline default size None on init Signed-off-by: ericharper <complex451@gmail.com> * make virtual pipeline default to None in config Signed-off-by: ericharper <complex451@gmail.com> * remove ensure_divisibility call Signed-off-by: ericharper <complex451@gmail.com> * fix lgtm alerts Signed-off-by: ericharper <complex451@gmail.com> * remove num_sanity_val_steps from config Signed-off-by: ericharper <complex451@gmail.com> * default virtual pipeline size to none Signed-off-by: ericharper <complex451@gmail.com> * check for list Signed-off-by: ericharper <complex451@gmail.com> * update assert to make sure we are only doing virtual for gpt Signed-off-by: ericharper <complex451@gmail.com> * revert change to get_params_for_weight_decay Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * init var Signed-off-by: ericharper <complex451@gmail.com> * add import guard for set virtual model parallel world size Signed-off-by: ericharper <complex451@gmail.com> * use import guard Signed-off-by: ericharper <complex451@gmail.com> * update calls to fake init in eval scripts Signed-off-by: ericharper <complex451@gmail.com> * add _get_fwd_bwd_function Signed-off-by: ericharper <complex451@gmail.com> * log all total model parameters Signed-off-by: ericharper <complex451@gmail.com> * remove unused import Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * reduced to 14 inactive days to be stale for PRs. (#5165) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com> * refactor TTS documentation organization and add new contents. (#5137) * refactor TTS documentation organization and add new contents. * fix asr api bug. * fix broken links. * fix unexpected indentation errors. * fixed unexpected indentation. * fixed broken paper reference. * fixed cross-reference and typos. * fixed toctree errors. * revert to 'Augmentors' * reordered TTS tutorial list in starthere. * ordered api classes alphabetically for each Section. * fixed underscore typo for fastpitch checkpoint. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * upcase 'Tuning' Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fixed typo for RAD-TTS Aligner Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * reorder aligner section after mel-gen and vocoders in models.rst. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * clarify Mixer-TTS-X and reorder model descriptions alphabetically. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fixed some typos and formats. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed old megatron.rst. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fixed block quote ends without a blank line warnings. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * remove duplicate reference; fixed missing key nlp-megatron-shoeybi2019megatron Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Revert "removed old megatron.rst." This reverts commit c5ea1dc3f23272eecfe8040e3abfa54fa122cf73. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed Russian, a hyphen, and add a note about G2P in tts/config.r…

* remove hardcoded input and output Signed-off-by: arendu <adithya.r@gmail.com> * fix inf table Signed-off-by: arendu <adithya.r@gmail.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com>

* [TTS] Add tutorial for TTS data prep scripts --------- Signed-off-by: Ryan <rlangman@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* aliases Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * add NEMO_PATH Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * expand_aliases Signed-off-by: Nikolay Karpov <karpnv@gmail.com> --------- Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

Signed-off-by: Igor Gitman <igitman@nvidia.com>

…ble models (#7012) (#7013) * [TTS] fastpitch: add english libritts model with asr stft parameters (25 ms 10 ms) * [TTS] enhancer: add pretrained model intended for asr finetuning --------- Signed-off-by: Roman Korostik <rkorostik@nvidia.com>

* Add ASR with TTS Tutorial * Fix enhancer usage Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com>

* Add end_strings to SamplingParams Signed-off-by: Gerald Shen <geshen@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Gerald Shen <geshen@nvidia.com> * Add end_strings to megatron_gpt_inference.yaml Signed-off-by: Gerald Shen <geshen@nvidia.com> * Add end_strings to sampling params Signed-off-by: Gerald Shen <geshen@nvidia.com> * Remove extra_id_1 from default end_strings Signed-off-by: Gerald Shen <geshen@nvidia.com> * Fix require_grad typos (#6930) Signed-off-by: Sergii Dymchenko <sdym@fb.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * fix syntax error Signed-off-by: Gerald Shen <geshen@nvidia.com> * fix the mpt chatbot (#6957) (#6968) Signed-off-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Yi Dong <43824965+yidong72@users.noreply.github.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * add support for max_total_length=4096 for 43b (#6763) * add support for max_total_length=4096 for 43b Signed-off-by: Zhilin Wang <wangzhilin12061996@hotmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * rnnt_greedy_decoding.py: typos? auto-repressively -> auto-regressively (#6989) Signed-off-by: Vadim Kantorov <vadimkantorov@gmail.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * Cache handling without input tensors mutation (#6980) (#6996) * Cache handling without input tensors mutation * Cleanup * Cleanup#2 * Cleanup#3 --------- Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * Hybrid conformer export (#6983) (#6995) * Implemented generic kv-pair setting of export_config from args * Hybrid conformer export * Hybrid decoder export * Cleanup * Changed from **kwargs * Docstring * Docs added * Stringify args * Added docs for ASR export configs * lowercase ctc --------- Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * Fixing an issue with confidence ensembles (#6987) (#7004) * Bug fix for the confidence ensembles * Relax constraints for the test --------- Signed-off-by: Igor Gitman <igitman@nvidia.com> Co-authored-by: Igor Gitman <igitman@nvidia.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * [TTS] Add cosine distance option to TTS aligner (#6806) * [TTS] Add cosine distance option to TTS aligner Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Update aligner comments Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * Minor MPT-7B fixes and creation script update (#6982) * Initial commit of minor MPT-7B fixes Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * Change Jenkins timeout (#6997) * change timeout Signed-off-by: ericharper <complex451@gmail.com> * change to 8 hours Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * remove hard coded input and output fields (#7008) * remove hard coded input and output fields Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * RoPE length extrapolation with interpolation (#7005) * Push changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * add continue training script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [WIP] nonlinear interp Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * override encoder_seq_len Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove nonlinear Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * sft with pi (#7006) * sft with pi Signed-off-by: Evelina <ebakhturina@nvidia.com> * update values only if not None" Signed-off-by: Evelina <ebakhturina@nvidia.com> --------- Signed-off-by: Evelina <ebakhturina@nvidia.com> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add info Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Empty Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> --------- Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Evelina <ebakhturina@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Signed-off-by: Gerald Shen <geshen@nvidia.com> * use proper config Signed-off-by: Gerald Shen <geshen@nvidia.com> * Add end_strings to SamplingParams Signed-off-by: Gerald Shen <geshen@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Gerald Shen <geshen@nvidia.com> * Add end_strings to megatron_gpt_inference.yaml Signed-off-by: Gerald Shen <geshen@nvidia.com> * Add end_strings to sampling params Signed-off-by: Gerald Shen <geshen@nvidia.com> * Remove extra_id_1 from default end_strings Signed-off-by: Gerald Shen <geshen@nvidia.com> * fix syntax error Signed-off-by: Gerald Shen <geshen@nvidia.com> * use proper config Signed-off-by: Gerald Shen <geshen@nvidia.com> --------- Signed-off-by: Gerald Shen <geshen@nvidia.com> Signed-off-by: Sergii Dymchenko <sdym@fb.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Signed-off-by: Vadim Kantorov <vadimkantorov@gmail.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Igor Gitman <igitman@nvidia.com> Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: Daniel Egert <degert@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Evelina <ebakhturina@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sergii Dymchenko <kit1980@gmail.com> Co-authored-by: Gerald Shen <geshen@nvidia.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Yi Dong <43824965+yidong72@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: Vadim Kantorov <vadimkantorov@gmail.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Igor Gitman <igitman@nvidia.com> Co-authored-by: Ryan Langman <rlangman@nvidia.com> Co-authored-by: trias702 <25867060+trias702@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com>

…es not wait for setup (#7016) Signed-off-by: Kim Ngo <6362111+findkim@users.noreply.github.com>

Signed-off-by: tbartley94 <tbartley@nvidia.com>

* rnnt_ngram_merge Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * char level bug Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Nikolay Karpov <karpnv@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>

Signed-off-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Yi Dong <43824965+yidong72@users.noreply.github.com>

* small fixes and tests Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * various fixes for the tutorial Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * tutorial added Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * for for a little oops after rebasement Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix tests Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * unused import removed Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * fix review comments Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * deprecated parameters for greedy configs Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * move re-assigning to configs Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * fix comments 2 Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * fix config tests Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * fix ece test (my env was bugged apparently) Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * renamings for confidence ensemble Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fox comments 3 Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * return dropped tutorial Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> * CI flips back and forth, increasing tolerance Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> --------- Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Signed-off-by: Nikolay Karpov <karpnv@gmail.com> Co-authored-by: Nikolay Karpov <karpnv@gmail.com>

Signed-off-by: Alexandra Antonova <antonova_sasha@list.ru> Co-authored-by: bene-ges <antonova_sasha@list.ru> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com>

Signed-off-by: Yi Dong <yidong@nvidia.com>

Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>

* st standalone model Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> * sacrebleu import fix, unused imports removed Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> * import guard for nlp inside asr transformer bpe model Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * codeql fixes Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * comments answered Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> * import ordering fix Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> * yttm for asr removed Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> * logging added Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> * added inference and translate method Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: AlexGrinch <grinchuk.alexey@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* remove pos emb from state dict Signed-off-by: Evelina <ebakhturina@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * move to nlp_model Signed-off-by: Evelina <ebakhturina@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update comment Signed-off-by: Evelina <ebakhturina@nvidia.com> * fix nmt test Signed-off-by: Evelina <ebakhturina@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix nmt test Signed-off-by: Evelina <ebakhturina@nvidia.com> --------- Signed-off-by: Evelina <ebakhturina@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

Signed-off-by: Vitaly Lavrukhin <vlavrukhin@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com>

* Fix documentation for Numba * Update force float32 flag dynamically * Update force float32 flag dynamically * Fix nemo version --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com>

* update fvad doc Signed-off-by: stevehuang52 <heh@nvidia.com> * fix typo Signed-off-by: stevehuang52 <heh@nvidia.com> * update fvad example Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * fix onnx export Signed-off-by: stevehuang52 <heh@nvidia.com> * update test Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * update doc Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>

* memmap worker arg Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Merge main to asr_normalize #7084

Merge main to asr_normalize #7084

Commits on Jun 1, 2023

Commits on Jun 2, 2023

Commits on Jun 3, 2023

Commits on Jun 4, 2023

Commits on Jun 5, 2023

Commits on Jun 6, 2023

Commits on Jun 7, 2023

Commits on Jun 8, 2023

Commits on Jun 9, 2023

Commits on Jun 10, 2023

Commits on Jun 12, 2023

Commits on Jun 13, 2023

Commits on Jun 14, 2023

Commits on Jun 15, 2023

Commits on Jun 17, 2023

Commits on Jun 18, 2023

Commits on Jun 19, 2023

Commits on Jun 20, 2023

Commits on Jun 21, 2023

Commits on Jun 22, 2023

Commits on Jun 23, 2023

Commits on Jun 25, 2023

Commits on Jun 26, 2023

Commits on Jun 27, 2023

Commits on Jun 28, 2023

Commits on Jun 29, 2023

Commits on Jun 30, 2023

Commits on Jul 1, 2023

Commits on Jul 3, 2023

Commits on Jul 5, 2023

Commits on Jul 7, 2023

Commits on Jul 10, 2023

Commits on Jul 11, 2023

Commits on Jul 12, 2023

Commits on Jul 13, 2023

Commits on Jul 14, 2023

Commits on Jul 15, 2023

Commits on Jul 17, 2023

Commits on Jul 18, 2023

Commits on Jul 19, 2023

Commits on Jul 20, 2023