-
-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI/Build] build on empty device for better dev experience #4773
Merged
youkaichao
merged 13 commits into
vllm-project:main
from
tomeras91:platform-agnostic-wheel
Aug 11, 2024
Merged
Changes from 10 commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
03850aa
mark xformers and vllm-flash-attn as installable only on x86_64 Linux
tomeras91 f47bc9c
changes in vllm to mock triton in case it can't be imported
tomeras91 bc83673
changes to setup.py to allow PLATFORM_AGNOSTIC_BUILD
tomeras91 947c2fa
format and lint fixes
tomeras91 24c513e
no need for triton installation in CPU requirements. Import errors ar…
tomeras91 3c1819c
Merge branch 'main' into platform-agnostic-wheel
tomeras91 c68f261
Merge branch 'vllm-project:main' into platform-agnostic-wheel
tomeras91 b77ea4c
Merge branch 'main' into platform-agnostic-wheel
tomeras91 cd0729f
reduce diffs from main
tomeras91 18b33ff
implement using VLLM_TARGET_DEVICE='empty'
tomeras91 f632c8e
make build of .tar.gz possible on mac
tomeras91 37c40ed
Add "+empty" to version for build with VLLM_TARGET_DEVICE="empty"
tomeras91 cf75037
(1) no f-strings in logs (2) warning instead of info log
tomeras91 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change is because we take the CUDA requirements for the "empty" device wheel, and
xformers
andvllm-flash-attn
are available only on LinuxThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
platform_system == 'Linux'
makes sense to me.is
platform_machine == 'x86_64'
necessary?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
vllm-flash-attn
have published wheels only forx86_64
, and no published tar.gz - https://pypi.org/project/vllm-flash-attn/#filesxformers
also has wheels only for 64bit machines. It does have a tar.gz but from what I found online it can't be installed on 32bit - https://pypi.org/project/xformers/#filesSo I'm pretty sure it's needed for both