Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multi groups for broadcast of sharding stage 2 #46894

Merged
merged 7 commits into from
Oct 12, 2022
Merged

Multi groups for broadcast of sharding stage 2 #46894

merged 7 commits into from
Oct 12, 2022

Conversation

FeixLiu
Copy link
Contributor

@FeixLiu FeixLiu commented Oct 11, 2022 •
edited
Loading

PR types

Others

PR changes

Others

Describe

Multi groups for broadcast of sharding stage 2

Loss

image

this is 2500 steps, 1 log means 10 steps

Speed

before pr gain
43507 44605 +2.5%

@paddle-bot
Copy link

paddle-bot bot commented Oct 11, 2022

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@FeixLiu FeixLiu merged commit 9576811 into PaddlePaddle:develop Oct 12, 2022
@FeixLiu FeixLiu deleted the multi_groups_for_broadcast branch October 12, 2022 03:13
FeixLiu added a commit to FeixLiu/Paddle that referenced this pull request Oct 17, 2022
@FeixLiu
Multi groups for broadcast of sharding stage 2 (PaddlePaddle#46894)
d45aa7d
@FeixLiu FeixLiu mentioned this pull request Oct 18, 2022
Merged
fuyinno4 pushed a commit that referenced this pull request Oct 18, 2022
@FeixLiu
Cherry pick for sharding (#47061)
5b64214 
* [dygraph sharding] Overlap the reduce and the caculation for sharding stage 2. (#46495)

* [dygraph sharding stage 2] sharding broadcast overlap (#46656)

* Multi groups for broadcast of sharding stage 2 (#46894)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sneaxiy sneaxiy sneaxiy approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@FeixLiu @sneaxiy