Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Log TFLOPS in NeMo-Megatron Models #4

Closed
wants to merge 40 commits into from
Closed

Log TFLOPS in NeMo-Megatron Models #4

wants to merge 40 commits into from

Commits on Sep 14, 2022

  1. update branch 

    Signed-off-by: ericharper <complex451@gmail.com>
    @ericharper
    ericharper committed Sep 14, 2022
    Configuration menu
    Copy the full SHA
    6d02e3a View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2022

  1. Add cherry-pick action (NVIDIA#4958) 

    * add cherry-pick action

Signed-off-by: ericharper <complex451@gmail.com>

* Pin Transformers version to fix CI (NVIDIA#4955)

* Pin transformers version in CI to prevent offline tokenizer loading error

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

* Drop version

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

* Disable offline temporarily

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

* Disable offline temporarily

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

* Enable offline

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: SeanNaren <snarenthiran@nvidia.com>
Co-authored-by: Sean Naren <snarenthiran@nvidia.com>
    @ericharper @SeanNaren
    ericharper and SeanNaren authored Sep 20, 2022
    Configuration menu
    Copy the full SHA
    1a8caa1 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2022

  1. Fix changelog builder (NVIDIA#4962) 

    Signed-off-by: smajumdar <smajumdar@nvidia.com>

Signed-off-by: smajumdar <smajumdar@nvidia.com>
    @titu1994
    titu1994 authored Sep 21, 2022
    Configuration menu
    Copy the full SHA
    7721947 View commit details
    Browse the repository at this point in the history

  2. fix cherry pick workflow (NVIDIA#4964) 

    Signed-off-by: ericharper <complex451@gmail.com>

Signed-off-by: ericharper <complex451@gmail.com>
    @ericharper
    ericharper authored Sep 21, 2022
    Configuration menu
    Copy the full SHA
    0e498ba View commit details
    Browse the repository at this point in the history

  3. reorder model check (NVIDIA#4959) 

    Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
    @nithinraok
    nithinraok authored Sep 21, 2022
    Configuration menu
    Copy the full SHA
    3f1295f View commit details
    Browse the repository at this point in the history

  4. check for active conda environment (NVIDIA#4970) 

    Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
    @nithinraok
    nithinraok authored Sep 21, 2022
    Configuration menu
    Copy the full SHA
    98473ba View commit details
    Browse the repository at this point in the history

  5. fix label models restoring issue from wrighted cross entropy (NVIDIA#… 

    …4968)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
    @nithinraok
    nithinraok authored Sep 21, 2022
    Configuration menu
    Copy the full SHA
    f49a47d View commit details
    Browse the repository at this point in the history

  6. [TTS] fix broken tutorial for MixerTTS. (NVIDIA#4949) 

    Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
    @XuesongYang
    XuesongYang authored Sep 21, 2022
    Configuration menu
    Copy the full SHA
    6616b04 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2022

  1. Fix ASR issues (NVIDIA#4984) 

    * Fix ASR issues

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Revert fix

Signed-off-by: smajumdar <smajumdar@nvidia.com>

Signed-off-by: smajumdar <smajumdar@nvidia.com>
    @titu1994
    titu1994 authored Sep 23, 2022
    Configuration menu
    Copy the full SHA
    a1f9067 View commit details
    Browse the repository at this point in the history

  2. Add simple pre-commit file (NVIDIA#4983) (NVIDIA#4995)

    @SeanNaren
    SeanNaren authored Sep 23, 2022
    Configuration menu
    Copy the full SHA
    76ae0fe View commit details
    Browse the repository at this point in the history

  3. [TTS] refactor aligner.encode into separate g2p process and encode_fr… 

    …om_g2p process to fix aligner bug. (NVIDIA#4992)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
    @XuesongYang
    XuesongYang authored Sep 23, 2022
    Configuration menu
    Copy the full SHA
    dccc95e View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2022

  1. move blossom ci to 22.08 

    Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
    @okuchaiev
    okuchaiev committed Sep 24, 2022
    Configuration menu
    Copy the full SHA
    a2cc90c View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2022

  1. Fix diarization ASR inference link in notebook (NVIDIA#5016) 

    * Fix diarization ASR inference link in notebook

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

* Update tutorials/speaker_tasks/ASR_with_SpeakerDiarization.ipynb

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>
    @SeanNaren
    SeanNaren authored Sep 27, 2022
    Configuration menu
    Copy the full SHA
    421f3b1 View commit details
    Browse the repository at this point in the history

  2. Fix bug in Squeezeformer Conv block (NVIDIA#5011) 

    * Fix bug in Squeezeformer Conv block

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Fix kernel context

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Fix access mixin

Signed-off-by: smajumdar <smajumdar@nvidia.com>

Signed-off-by: smajumdar <smajumdar@nvidia.com>
    @titu1994
    titu1994 authored Sep 27, 2022
    Configuration menu
    Copy the full SHA
    fd64656 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2022

  1. Fix decoding bug for megatron enc-dec models with O2 (NVIDIA#4989) 

    * Change inspect

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix cross attention RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
    @MaximumEntropy @okuchaiev
    MaximumEntropy and okuchaiev authored Sep 28, 2022
    Configuration menu
    Copy the full SHA
    cc8a1b8 View commit details
    Browse the repository at this point in the history

  2. upper bound transformers 

    Signed-off-by: ericharper <complex451@gmail.com>
    @ericharper
    ericharper committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    84e8421 View commit details
    Browse the repository at this point in the history

  3. remove duplicate transformers requirement 

    Signed-off-by: ericharper <complex451@gmail.com>
    @ericharper
    ericharper committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    0f08de1 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2022

  1. Fix bugs (NVIDIA#5036)

    @Zhilin123
    Zhilin123 authored Sep 29, 2022
    Configuration menu
    Copy the full SHA
    a036287 View commit details
    Browse the repository at this point in the history

  2. Fix zh tn (NVIDIA#5035) 

    * add missing files for package

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add core as dependecy to nemo_text_processing

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
    @yzhang123
    yzhang123 authored Sep 29, 2022
    Configuration menu
    Copy the full SHA
    1ce05da View commit details
    Browse the repository at this point in the history

  3. Add black to pre-commit (NVIDIA#5027) (NVIDIA#5045) 

    Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>
(cherry picked from commit 986a766)
    @SeanNaren
    SeanNaren authored Sep 29, 2022
    Configuration menu
    Copy the full SHA
    23fbe69 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2022

  1. Update Question_Answering.ipynb (NVIDIA#5049)

    @Zhilin123
    Zhilin123 authored Sep 30, 2022
    Configuration menu
    Copy the full SHA
    d7b19c1 View commit details
    Browse the repository at this point in the history

  2. Bug fixes for parallel mp3 to wav conversion, PC notebook, update Rea… 

    …dme for TN requirements (NVIDIA#5047)

* bug fixes segmenation, pc

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* bug fixes segmenation, pc

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* install pynini

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* add requirements install back

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* update sox requirements

Signed-off-by: ekmb <ebakhturina@nvidia.com>

Signed-off-by: ekmb <ebakhturina@nvidia.com>
    @ekmb
    ekmb authored Sep 30, 2022
    Configuration menu
    Copy the full SHA
    3244554 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2022

  1. Missing fixes from r1.11.0 to T5 finetuning eval (NVIDIA#5054) 

    * Fixes to seq2seq eval

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    @MaximumEntropy @pre-commit-ci
    MaximumEntropy and pre-commit-ci[bot] authored Oct 3, 2022
    Configuration menu
    Copy the full SHA
    e85b2a7 View commit details
    Browse the repository at this point in the history

  2. fixed megatron lm conversion bug (PTL related) (NVIDIA#5038) 

    Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>

Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>
Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com>
    @Davood-M
    Davood-M and Davood-M authored Oct 3, 2022
    Configuration menu
    Copy the full SHA
    316dd27 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2022

  1. P&C Docs (NVIDIA#5068) 

    Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>

Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>
    @jubick1337
    jubick1337 authored Oct 4, 2022
    Configuration menu
    Copy the full SHA
    961e5b6 View commit details
    Browse the repository at this point in the history

  2. [TTS] added missing German phoneme tokenizer. (NVIDIA#5070) 

    Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
    @XuesongYang
    XuesongYang authored Oct 4, 2022
    Configuration menu
    Copy the full SHA
    86eec01 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2022

  1. probabilites -> probabilities (NVIDIA#5078) 

    Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
    @nithinraok
    nithinraok authored Oct 5, 2022
    Configuration menu
    Copy the full SHA
    429aea0 View commit details
    Browse the repository at this point in the history

  2. [TTS] remove phonemizer.py (NVIDIA#5071) 

    * [TTS] remove phonemizer.py
* convert code block to markdown.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
    @XuesongYang
    XuesongYang authored Oct 5, 2022
    Configuration menu
    Copy the full SHA
    2b28d8b View commit details
    Browse the repository at this point in the history

  3. Release SOTA Lang ID model (NVIDIA#5080) 

    * add pretrained lang id model ambernet

Signed-off-by: fayejf <fayejf07@gmail.com>

* update doc and style fix

Signed-off-by: fayejf <fayejf07@gmail.com>

Signed-off-by: fayejf <fayejf07@gmail.com>
    @fayejf
    fayejf authored Oct 5, 2022
    Configuration menu
    Copy the full SHA
    36654e7 View commit details
    Browse the repository at this point in the history

  4. Notebook bug fixes (NVIDIA#5084) 

    * Notebook bug fixes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Turned nemo install back on

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* reverted notebook

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Updated one line in entity linking nb

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>
Co-authored-by: Eric Harper <complex451@gmail.com>
    @vadam5 @ericharper
    vadam5 and ericharper authored Oct 5, 2022
    Configuration menu
    Copy the full SHA
    56b4d85 View commit details
    Browse the repository at this point in the history

  5. update strategy in notebook from ddp_fork to dp (NVIDIA#5088)

    @Zhilin123
    Zhilin123 authored Oct 5, 2022
    Configuration menu
    Copy the full SHA
    1938f3d View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2022

  1. Fix Unhashable type list for Numba Cuda spec augment kernel (NVIDIA#5093 

    )

Signed-off-by: smajumdar <smajumdar@nvidia.com>

Signed-off-by: smajumdar <smajumdar@nvidia.com>
    @titu1994
    titu1994 authored Oct 6, 2022
    Configuration menu
    Copy the full SHA
    ed8433c View commit details
    Browse the repository at this point in the history

  2. Remove numba import (NVIDIA#5095) 

    Signed-off-by: smajumdar <smajumdar@nvidia.com>

Signed-off-by: smajumdar <smajumdar@nvidia.com>
    @titu1994
    titu1994 authored Oct 6, 2022
    Configuration menu
    Copy the full SHA
    520b341 View commit details
    Browse the repository at this point in the history

  3. T5 prompt learning fixes missing from r.11.0 merge (NVIDIA#5075) 

    * Fix special tokens

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Empty

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Co-authored-by: David <amosalla@asu.edu>
    @MaximumEntropy @Davood-M
    MaximumEntropy and Davood-M authored Oct 6, 2022
    Configuration menu
    Copy the full SHA
    f9e919f View commit details
    Browse the repository at this point in the history

  4. [TTS] fixed wrong val loss for epoch 0 and inconsistent metrics names (… 

    …NVIDIA#5087)

* fixed hifigan configs as well
* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    @XuesongYang @pre-commit-ci
    XuesongYang and pre-commit-ci[bot] authored Oct 6, 2022
    Configuration menu
    Copy the full SHA
    fcd0709 View commit details
    Browse the repository at this point in the history

  5. T5 Decoding with PP > 2 fix (NVIDIA#5091) 

    * set sequence lenghts in the pipeline properly

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
    @MaximumEntropy
    MaximumEntropy authored Oct 6, 2022
    Configuration menu
    Copy the full SHA
    a4069e0 View commit details
    Browse the repository at this point in the history

  6. Multiprocessing fix (NVIDIA#5106) 

    Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>

Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>
    @jubick1337
    jubick1337 authored Oct 6, 2022
    Configuration menu
    Copy the full SHA
    9b6a7bc View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2022

  1. [Bug fix] PC lexical + audio (NVIDIA#5109) 

    * training running

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* revert

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* revert

Signed-off-by: ekmb <ebakhturina@nvidia.com>

Signed-off-by: ekmb <ebakhturina@nvidia.com>
    @ekmb
    ekmb authored Oct 7, 2022
    Configuration menu
    Copy the full SHA
    c0fc759 View commit details
    Browse the repository at this point in the history

  2. bugfix: pybtex.database.InvalidNameString: Too many commas in author … 

    …field. (NVIDIA#5112)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
    @XuesongYang
    XuesongYang authored Oct 7, 2022
    Configuration menu
    Copy the full SHA
    dd9a30f View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2022

  1. Add @QuentinAnthony TFLOPS logging code. 

    Co-authored-by: Quentin Anthony <anthony.301@osu.edu>
Signed-off-by: Dashiell Stander <dstander@protonmail.com>
    @dashstander @Quentin-Anthony
    dashstander and Quentin-Anthony committed Dec 16, 2022
    Configuration menu
    Copy the full SHA
    90126de View commit details
    Browse the repository at this point in the history