Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Faster OSS #42

Closed
blefaudeux opened this issue Aug 14, 2020 · 0 comments · Fixed by #47
Closed

Faster OSS #42

blefaudeux opened this issue Aug 14, 2020 · 0 comments · Fixed by #47
Assignees
@blefaudeux

Comments

@blefaudeux
Copy link
Contributor

🚀 Feature

Reduce the runtime cost tied to activating OSS

Motivation

Right now the speed penalty of using OSS over a couple of nodes, for a typical ClassyVision load is 2-10x.

Pitch

There are a bunch of low hanging fruits, from oss_ddp to broadcast batching which could be used to reduce this impact

Alternatives

Not using OSS I guess
@blefaudeux blefaudeux self-assigned this Aug 14, 2020
This was referenced Aug 14, 2020
Closed
Merged
This was linked to pull requests Aug 20, 2020
[feat] batch broadcast requests into a configurable buffer #43
Closed
[feat] Simple macro OSS benchmark #47
Merged
myleott added a commit that referenced this issue Feb 22, 2021
@myleott
Only sync fp32_to_fp16 stream for the top-most (root) ShardParams wra…
cbd243e 
…pper (#42)

* Only sync fp32_to_fp16 stream for the top-most (root) ShardParams wrapper

* Fix mypy, add test, address some comments

* Add missing assert

* Comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@blefaudeux blefaudeux

Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@blefaudeux