Log TFLOPS in NeMo-Megatron Models #4
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: ericharper <complex451@gmail.com>
* add cherry-pick action Signed-off-by: ericharper <complex451@gmail.com> * Pin Transformers version to fix CI (NVIDIA#4955) * Pin transformers version in CI to prevent offline tokenizer loading error Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Drop version Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Disable offline temporarily Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Disable offline temporarily Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Enable offline Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com>
Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com>
Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
…4968) Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
* Fix ASR issues Signed-off-by: smajumdar <smajumdar@nvidia.com> * Revert fix Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com>
…om_g2p process to fix aligner bug. (NVIDIA#4992) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
* Fix diarization ASR inference link in notebook Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Update tutorials/speaker_tasks/ASR_with_SpeakerDiarization.ipynb Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com>
* Fix bug in Squeezeformer Conv block Signed-off-by: smajumdar <smajumdar@nvidia.com> * Fix kernel context Signed-off-by: smajumdar <smajumdar@nvidia.com> * Fix access mixin Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com>
* Change inspect Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix cross attention RPE Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: ericharper <complex451@gmail.com>
* add missing files for package Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * add core as dependecy to nemo_text_processing Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> (cherry picked from commit 986a766)
…dme for TN requirements (NVIDIA#5047) * bug fixes segmenation, pc Signed-off-by: ekmb <ebakhturina@nvidia.com> * bug fixes segmenation, pc Signed-off-by: ekmb <ebakhturina@nvidia.com> * install pynini Signed-off-by: ekmb <ebakhturina@nvidia.com> * add requirements install back Signed-off-by: ekmb <ebakhturina@nvidia.com> * update sox requirements Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com>
* Fixes to seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com>
Signed-off-by: Matvei Novikov <mattyson.so@gmail.com> Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
* [TTS] remove phonemizer.py * convert code block to markdown. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
* add pretrained lang id model ambernet Signed-off-by: fayejf <fayejf07@gmail.com> * update doc and style fix Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: fayejf <fayejf07@gmail.com>
* Notebook bug fixes Signed-off-by: Virginia Adams <vadams@nvidia.com> * Turned nemo install back on Signed-off-by: Virginia Adams <vadams@nvidia.com> * reverted notebook Signed-off-by: Virginia Adams <vadams@nvidia.com> * Updated one line in entity linking nb Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com>
Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com>
* Fix special tokens Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Empty Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: David <amosalla@asu.edu>
…NVIDIA#5087) * fixed hifigan configs as well * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* set sequence lenghts in the pipeline properly Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: Matvei Novikov <mattyson.so@gmail.com> Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>
* training running Signed-off-by: ekmb <ebakhturina@nvidia.com> * revert Signed-off-by: ekmb <ebakhturina@nvidia.com> * revert Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com>
…field. (NVIDIA#5112) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Quentin Anthony <anthony.301@osu.edu> Signed-off-by: Dashiell Stander <dstander@protonmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I took the liberty of copying @Quentin-Anthony 's PR to add TFLOPS logging to the Megatron model. This way it can get merged into main separately from #1 and hopefully clean up that PR even further.