Cherry Pick T5 finetuning changes into 1.13

[MaximumEntropy]

…tch size refactoring (#5339)

• Initial refactor

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• Resolve config before passing to load_from_checkpoint

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• Fixes for model parallel and nemo restore

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• Fixes for eval

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

• Revert config changes

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• Refactor

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

• Fix typo

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• Remove comments

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• Minor

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

• Fix validation reconfiguration

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• Remove old comment

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

• Fixes for test_ds

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca

• [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: MaximumEntropy sandeep.subramanian.1@umontreal.ca
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Collection: [Note which collection this PR will affect]

Changelog

• Add specific line by line info of high level changes in this PR.

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this 

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• New Feature
• Bugfix
• Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

• Related to # (issue)

[Davood-M]

LGTM, thank you!

[lgtm-com]

This pull request introduces 5 alerts when merging 1358e89 into 4a523ad - view on LGTM.com

new alerts:

• 3 for Wrong number of arguments in a call
• 1 for Unused local variable
• 1 for Unused import

---

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. It looks like GitHub code scanning with CodeQL is already set up for this repo, so no further action is needed 🚀. For more information, please check out our post on the GitHub blog.