Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use GPTModel from mcore #7093

Merged
merged 78 commits into from
Aug 14, 2023
Merged

Use GPTModel from mcore #7093

merged 78 commits into from
Aug 14, 2023

Conversation

ericharper
Copy link
Collaborator

@ericharper ericharper commented Jul 21, 2023 •
edited
Loading

What does this PR do ?

This PR adds a path to use GPTModel from Megatron Core.

Requirements:
Make sure that megatron core is installed from a recent commit:

git clone https://github.com/NVIDIA/Megatron-LM.git && \
            cd Megatron-LM && \
            git checkout 3316e811cc5335ee24c2d203416d864edcf2f7a8 && \
            pip install -e .

Collection: NLP

Changelog

  • Add specific line by line info of high level changes in this PR.

Usage

Set mcore_gpt=True in the MegatronGPTModel config.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

  • Related to # (issue)

@github-actions github-actions bot added core Changes to NeMo Core NLP CI labels Jul 21, 2023
github-advanced-security[bot]
github-advanced-security bot found potential problems Jul 21, 2023
View reviewed changes
Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

@ericharper ericharper marked this pull request as ready for review July 25, 2023 18:16
@ericharper ericharper changed the base branch from main to mcore_gpt_path July 25, 2023 18:16
@github-actions github-actions bot removed core Changes to NeMo Core CI labels Jul 25, 2023
ericharper and others added 21 commits August 8, 2023 10:46
@ericharper
start adding gpt from megatron core path
63c127d 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
set model parallel config
16d85c4 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
use model parallel config object
19e1420 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
update args
a309c0b 
Signed-off-by: ericharper <complex451@gmail.com>
@pre-commit-ci @ericharper
[pre-commit.ci] auto fixes from pre-commit.com hooks
2ea9285 
for more information, see https://pre-commit.ci
@ericharper
set vp size to none if it is 1
46ec121 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
set vp size to none if it is 1
575ef8a 
Signed-off-by: ericharper <complex451@gmail.com>
@pre-commit-ci @ericharper
[pre-commit.ci] auto fixes from pre-commit.com hooks
a8b177c 
for more information, see https://pre-commit.ci
@ericharper
add TransformerConfig
a296be8 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
start updating to TransformerConfig
ec3c170 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
add todo
e2090ae 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
revert to model parallel config
e1f38d8 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
add hidden_size to model_parallel_config
cbfb0d4 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
remove imports
cbf5036 
Signed-off-by: ericharper <complex451@gmail.com>
@pre-commit-ci @ericharper
[pre-commit.ci] auto fixes from pre-commit.com hooks
c6fe7ed 
for more information, see https://pre-commit.ci
@ericharper
remove import
2bd408c 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
small clean up
06064bf 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
update hidden size in peft base model, add mcore commit to jenkins
d8e9f4f 
Signed-off-by: ericharper <complex451@gmail.com>
@ericharper
update module args
afdf3f0 
Signed-off-by: ericharper <complex451@gmail.com>
@pre-commit-ci @ericharper
[pre-commit.ci] auto fixes from pre-commit.com hooks
90e8160 
for more information, see https://pre-commit.ci
@ericharper
add config obj to flash attention tests
3f194ac 
Signed-off-by: ericharper <complex451@gmail.com>
ericharper and others added 4 commits August 10, 2023 11:29
@ericharper
set virtual rank
6fb875b 
Signed-off-by: ericharper <complex451@gmail.com>
@pre-commit-ci @ericharper
[pre-commit.ci] auto fixes from pre-commit.com hooks
299e7b5 
for more information, see https://pre-commit.ci
@blahBlahhhJ @ericharper
Add GQA config to megatron gpt model (#7096)
d4a7576 
* Add GQA config in gpt config file

Signed-off-by: jasonwan <jasonwan@nvidia.com>

* Verify mcore is enabled when using GQA

Signed-off-by: jasonwan <jasonwan@nvidia.com>

---------

Signed-off-by: jasonwan <jasonwan@nvidia.com>
@ericharper
revert
3213cac 
Signed-off-by: ericharper <complex451@gmail.com>
ericharper and others added 8 commits August 10, 2023 12:27
Base automatically changed from mcore_gpt_path to main August 14, 2023 04:55
@ericharper
pull main
5105658 
Signed-off-by: eharper <eharper@nvidia.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Aug 14, 2023
View reviewed changes
nemo/collections/nlp/modules/common/megatron/adapters/parallel_adapters.py Fixed Show fixed Hide fixed
nemo/collections/nlp/models/language_modeling/megatron_base_model.py Fixed Show fixed Hide fixed
nemo/collections/nlp/parts/utils_funcs.py
elif activation == 'squared-relu':
activation_func = squared_relu

return activation_func

Check failure

Code scanning / CodeQL

Potentially uninitialized local variable

Local variable 'activation_func' may be used before it is initialized.
nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py Fixed Show fixed Hide fixed
@ericharper
update megatron core commit
9808ec4 
Signed-off-by: eharper <eharper@nvidia.com>
@github-actions github-actions bot added the CI label Aug 14, 2023
ericharper and others added 5 commits August 14, 2023 11:56
@ericharper
revert change
5a20f0d 
Signed-off-by: eharper <eharper@nvidia.com>
@ericharper
remove import
d5da981 
Signed-off-by: eharper <eharper@nvidia.com>
@ericharper
remove import
dbdffc6 
Signed-off-by: eharper <eharper@nvidia.com>
@ericharper
remove import
61a0902 
Signed-off-by: eharper <eharper@nvidia.com>
@ericharper
Merge branch 'main' into mcore_gpt_model
cb4850d
aklife97
aklife97 reviewed Aug 14, 2023
View reviewed changes
examples/nlp/language_modeling/conf/megatron_gpt_config.yaml
@@ -44,6 +44,9 @@ exp_manager:
model_parallel_size: ${multiply:${model.tensor_model_parallel_size}, ${model.pipeline_model_parallel_size}}

model:
# use GPTModel from megatron.core
mcore_gpt: False
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add a CI test for mcore gpt = True

aklife97
aklife97 approved these changes Aug 14, 2023
View reviewed changes
Copy link
Collaborator

@aklife97 aklife97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ericharper ericharper merged commit e64076e into main Aug 14, 2023
@ericharper ericharper deleted the mcore_gpt_model branch August 14, 2023 21:37
dorotat-nv pushed a commit to dorotat-nv/NeMo that referenced this pull request Aug 24, 2023
@ericharper @pre-commit-ci
@blahBlahhhJ @dorotat-nv
Use GPTModel from mcore (NVIDIA#7093)
2bf121d 
* start adding gpt from megatron core path

Signed-off-by: ericharper <complex451@gmail.com>

* set model parallel config

Signed-off-by: ericharper <complex451@gmail.com>

* use model parallel config object

Signed-off-by: ericharper <complex451@gmail.com>

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* set vp size to none if it is 1

Signed-off-by: ericharper <complex451@gmail.com>

* set vp size to none if it is 1

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add TransformerConfig

Signed-off-by: ericharper <complex451@gmail.com>

* start updating to TransformerConfig

Signed-off-by: ericharper <complex451@gmail.com>

* add todo

Signed-off-by: ericharper <complex451@gmail.com>

* revert to model parallel config

Signed-off-by: ericharper <complex451@gmail.com>

* add hidden_size to model_parallel_config

Signed-off-by: ericharper <complex451@gmail.com>

* remove imports

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove import

Signed-off-by: ericharper <complex451@gmail.com>

* small clean up

Signed-off-by: ericharper <complex451@gmail.com>

* update hidden size in peft base model, add mcore commit to jenkins

Signed-off-by: ericharper <complex451@gmail.com>

* update module args

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add config obj to flash attention tests

Signed-off-by: ericharper <complex451@gmail.com>

* remove args

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove sequence parallel arg

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* add config to self

Signed-off-by: ericharper <complex451@gmail.com>

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* add config to test

Signed-off-by: ericharper <complex451@gmail.com>

* get hidden_size from config

Signed-off-by: ericharper <complex451@gmail.com>

* add try except

Signed-off-by: ericharper <complex451@gmail.com>

* use default

Signed-off-by: ericharper <complex451@gmail.com>

* update config with hidden size

Signed-off-by: ericharper <complex451@gmail.com>

* remove arg

Signed-off-by: ericharper <complex451@gmail.com>

* comment out jenkins test

Signed-off-by: ericharper <complex451@gmail.com>

* revert import

Signed-off-by: ericharper <complex451@gmail.com>

* remove optimizer_idx

Signed-off-by: eharper <eharper@nvidia.com>

* prefetch num microbatches

Signed-off-by: eharper <eharper@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* start adding gpt from megatron core path

Signed-off-by: ericharper <complex451@gmail.com>

* set model parallel config

Signed-off-by: ericharper <complex451@gmail.com>

* use model parallel config object

Signed-off-by: ericharper <complex451@gmail.com>

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* start updating to TransformerConfig

Signed-off-by: ericharper <complex451@gmail.com>

* revert to model parallel config

Signed-off-by: ericharper <complex451@gmail.com>

* add hidden_size to model_parallel_config

Signed-off-by: ericharper <complex451@gmail.com>

* remove imports

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update module args

Signed-off-by: ericharper <complex451@gmail.com>

* add config to self

Signed-off-by: ericharper <complex451@gmail.com>

* build transformer config

Signed-off-by: ericharper <complex451@gmail.com>

* add model to provider func

Signed-off-by: ericharper <complex451@gmail.com>

* update forward and float16 wrapper

Signed-off-by: ericharper <complex451@gmail.com>

* instantiate model parallel config after init model parallel

Signed-off-by: ericharper <complex451@gmail.com>

* set virtual rank

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add GQA config to megatron gpt model (NVIDIA#7096)

* Add GQA config in gpt config file

Signed-off-by: jasonwan <jasonwan@nvidia.com>

* Verify mcore is enabled when using GQA

Signed-off-by: jasonwan <jasonwan@nvidia.com>

---------

Signed-off-by: jasonwan <jasonwan@nvidia.com>

* revert

Signed-off-by: ericharper <complex451@gmail.com>

* remove import

Signed-off-by: eharper <eharper@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update for dist adam

Signed-off-by: eharper <eharper@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* use get_gpt_module_list

Signed-off-by: eharper <eharper@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update megatron core commit

Signed-off-by: eharper <eharper@nvidia.com>

* revert change

Signed-off-by: eharper <eharper@nvidia.com>

* remove import

Signed-off-by: eharper <eharper@nvidia.com>

* remove import

Signed-off-by: eharper <eharper@nvidia.com>

* remove import

Signed-off-by: eharper <eharper@nvidia.com>

---------

Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: eharper <eharper@nvidia.com>
Signed-off-by: jasonwan <jasonwan@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jason Wang <jasonwan@nvidia.com>
Signed-off-by: dorotat <dorotat@nvidia.com>
@timmoon10 timmoon10 mentioned this pull request Apr 2, 2024
Merged
8 tasks
rohitrango pushed a commit to rohitrango/NeMo that referenced this pull request Jun 25, 2024
@ericharper @pre-commit-ci @blahBlahhhJ
Use GPTModel from mcore (NVIDIA#7093)
af21e99 
* start adding gpt from megatron core path

Signed-off-by: ericharper <complex451@gmail.com>

* set model parallel config

Signed-off-by: ericharper <complex451@gmail.com>

* use model parallel config object

Signed-off-by: ericharper <complex451@gmail.com>

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* set vp size to none if it is 1

Signed-off-by: ericharper <complex451@gmail.com>

* set vp size to none if it is 1

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add TransformerConfig

Signed-off-by: ericharper <complex451@gmail.com>

* start updating to TransformerConfig

Signed-off-by: ericharper <complex451@gmail.com>

* add todo

Signed-off-by: ericharper <complex451@gmail.com>

* revert to model parallel config

Signed-off-by: ericharper <complex451@gmail.com>

* add hidden_size to model_parallel_config

Signed-off-by: ericharper <complex451@gmail.com>

* remove imports

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove import

Signed-off-by: ericharper <complex451@gmail.com>

* small clean up

Signed-off-by: ericharper <complex451@gmail.com>

* update hidden size in peft base model, add mcore commit to jenkins

Signed-off-by: ericharper <complex451@gmail.com>

* update module args

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add config obj to flash attention tests

Signed-off-by: ericharper <complex451@gmail.com>

* remove args

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove sequence parallel arg

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* add config to self

Signed-off-by: ericharper <complex451@gmail.com>

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* add config to test

Signed-off-by: ericharper <complex451@gmail.com>

* get hidden_size from config

Signed-off-by: ericharper <complex451@gmail.com>

* add try except

Signed-off-by: ericharper <complex451@gmail.com>

* use default

Signed-off-by: ericharper <complex451@gmail.com>

* update config with hidden size

Signed-off-by: ericharper <complex451@gmail.com>

* remove arg

Signed-off-by: ericharper <complex451@gmail.com>

* comment out jenkins test

Signed-off-by: ericharper <complex451@gmail.com>

* revert import

Signed-off-by: ericharper <complex451@gmail.com>

* remove optimizer_idx

Signed-off-by: eharper <eharper@nvidia.com>

* prefetch num microbatches

Signed-off-by: eharper <eharper@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* start adding gpt from megatron core path

Signed-off-by: ericharper <complex451@gmail.com>

* set model parallel config

Signed-off-by: ericharper <complex451@gmail.com>

* use model parallel config object

Signed-off-by: ericharper <complex451@gmail.com>

* update args

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* start updating to TransformerConfig

Signed-off-by: ericharper <complex451@gmail.com>

* revert to model parallel config

Signed-off-by: ericharper <complex451@gmail.com>

* add hidden_size to model_parallel_config

Signed-off-by: ericharper <complex451@gmail.com>

* remove imports

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update module args

Signed-off-by: ericharper <complex451@gmail.com>

* add config to self

Signed-off-by: ericharper <complex451@gmail.com>

* build transformer config

Signed-off-by: ericharper <complex451@gmail.com>

* add model to provider func

Signed-off-by: ericharper <complex451@gmail.com>

* update forward and float16 wrapper

Signed-off-by: ericharper <complex451@gmail.com>

* instantiate model parallel config after init model parallel

Signed-off-by: ericharper <complex451@gmail.com>

* set virtual rank

Signed-off-by: ericharper <complex451@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add GQA config to megatron gpt model (NVIDIA#7096)

* Add GQA config in gpt config file

Signed-off-by: jasonwan <jasonwan@nvidia.com>

* Verify mcore is enabled when using GQA

Signed-off-by: jasonwan <jasonwan@nvidia.com>

---------

Signed-off-by: jasonwan <jasonwan@nvidia.com>

* revert

Signed-off-by: ericharper <complex451@gmail.com>

* remove import

Signed-off-by: eharper <eharper@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update for dist adam

Signed-off-by: eharper <eharper@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* use get_gpt_module_list

Signed-off-by: eharper <eharper@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update megatron core commit

Signed-off-by: eharper <eharper@nvidia.com>

* revert change

Signed-off-by: eharper <eharper@nvidia.com>

* remove import

Signed-off-by: eharper <eharper@nvidia.com>

* remove import

Signed-off-by: eharper <eharper@nvidia.com>

* remove import

Signed-off-by: eharper <eharper@nvidia.com>

---------

Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: eharper <eharper@nvidia.com>
Signed-off-by: jasonwan <jasonwan@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jason Wang <jasonwan@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aklife97 aklife97 aklife97 approved these changes

Assignees
No one assigned
Labels
CI NLP
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ericharper @aklife97 @blahBlahhhJ