-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add wheel builds to cuxfilter #497
Merged
Merged
Changes from all commits
Commits
Show all changes
58 commits
Select commit
Hold shift + click to select a range
93e1471
update to cuda12
AjayThorve 952d660
migrate to pyproject.toml
AjayThorve 2a13105
add gha to build, publish and test wheels
AjayThorve 9736d17
add new line at the end
AjayThorve 595082f
add apply_wheel_modifications script
AjayThorve 10dcf41
rm README file
AjayThorve 964ebbd
Add symbolic link to README file to fix pyproject.toml referencing a …
AjayThorve 1198848
fix typo
AjayThorve b315075
Update .pre-commit-config.yaml
AjayThorve 49d3c88
update to cuda12
AjayThorve 9690c3f
fix pyproject.toml path
AjayThorve 6e92fa8
update dependencies.yaml to include pyproject.toml
AjayThorve 167ea9b
version conditions in pep508
AjayThorve ebe0d1e
rm manifest and _version
AjayThorve 3c794bf
fix versions
AjayThorve f975324
add newline at end
AjayThorve 8194361
fix dependencies
AjayThorve 91f258e
wheels for CUDA 12 use cupy-cuda12x
AjayThorve 582b416
fix modification scripts for typos
AjayThorve e6fa115
Merge branch 'branch-23.08' of github.com:rapidsai/cuxfilter into add…
AjayThorve efd2f4f
use wheels-pure workflows for pure python builds
AjayThorve 07ccd5c
remove incorrect input
AjayThorve 238581a
update to cuda12
AjayThorve 7561383
migrate to pyproject.toml
AjayThorve f8e992e
add gha to build, publish and test wheels
AjayThorve 343e4b7
add new line at the end
AjayThorve 580f1c0
add apply_wheel_modifications script
AjayThorve 9f1d287
rm README file
AjayThorve 000969c
Add symbolic link to README file to fix pyproject.toml referencing a …
AjayThorve e64915f
fix typo
AjayThorve d3ee729
Update .pre-commit-config.yaml
AjayThorve fb40b09
update to cuda12
AjayThorve d124ee1
fix pyproject.toml path
AjayThorve f20cb8c
update dependencies.yaml to include pyproject.toml
AjayThorve ab01fd3
version conditions in pep508
AjayThorve aab4898
rm manifest and _version
AjayThorve 0753fc3
fix versions
AjayThorve 01b9c4b
add newline at end
AjayThorve 4ac9db1
fix dependencies
AjayThorve ca1919c
wheels for CUDA 12 use cupy-cuda12x
AjayThorve 991719c
fix modification scripts for typos
AjayThorve 0dd4755
Merge branch 'branch-23.08' of github.com:rapidsai/cuxfilter into add…
AjayThorve 3a674f3
use wheels-pure workflows for pure python builds
AjayThorve 67a2697
remove incorrect input
AjayThorve b517de1
Update .github/workflows/build.yaml
AjayThorve 940230e
Update .github/workflows/build.yaml
AjayThorve 7543e17
Update .github/workflows/pr.yaml
AjayThorve b41d1d4
Update .flake8
AjayThorve 630185f
Merge branch 'branch-23.08' of github.com:rapidsai/cuxfilter into add…
AjayThorve f9cd11f
Merge branch 'add-pip-wheels' of github.com:AjayThorve/cuxfilter into…
AjayThorve 3e45265
undo version usage, and use minor_version
AjayThorve ccbd94d
Apply suggestions from code review
AjayThorve 8c3556c
fix dependencies
AjayThorve ad01bad
update versions, using `pre-commit autoupdate`
AjayThorve d5c393e
run updated black
AjayThorve 26e4712
install dask-cuda from source
AjayThorve 6574a6c
remove dask-cuda as a direct dependency, since it's only required for…
AjayThorve a155801
use pytest-xdist extension
AjayThorve File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
# Copyright (c) 2023, NVIDIA CORPORATION. | ||
|
||
[flake8] | ||
filename = *.py, | ||
exclude = __init__.py, *.egg, build, docs, .git | ||
ignore = | ||
# line break before binary operator | ||
W503, | ||
# whitespace before : | ||
E203 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
#!/bin/bash | ||
# Copyright (c) 2023, NVIDIA CORPORATION. | ||
# | ||
# Usage: bash apply_wheel_modifications.sh <new_version> <cuda_suffix> | ||
|
||
VERSION=${1} | ||
CUDA_SUFFIX=${2} | ||
|
||
# pyproject.toml versions | ||
sed -i "s/^version = .*/version = \"${VERSION}\"/g" python/pyproject.toml | ||
|
||
# cuxfilter pyproject.toml cuda suffixes | ||
sed -i "s/^name = \"cuxfilter\"/name = \"cuxfilter${CUDA_SUFFIX}\"/g" python/pyproject.toml | ||
# Need to provide the == to avoid modifying the URL | ||
sed -i "s/\"cudf==/\"cudf${CUDA_SUFFIX}==/g" python/pyproject.toml | ||
sed -i "s/\"cuspatial==/\"cuspatial${CUDA_SUFFIX}==/g" python/pyproject.toml | ||
sed -i "s/\"dask-cudf==/\"dask-cudf${CUDA_SUFFIX}==/g" python/pyproject.toml | ||
|
||
if [[ $CUDA_SUFFIX == "-cu12" ]]; then | ||
sed -i "s/cupy-cuda11x/cupy-cuda12x/g" python/pyproject.toml | ||
fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
import cuxfilter | ||
|
||
if __name__ == '__main__': | ||
assert cuxfilter.__version__ is not None |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../README.md |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
# Copyright (c) 2019-2023, NVIDIA CORPORATION. | ||
|
||
from cuxfilter.dataframe import DataFrame | ||
AjayThorve marked this conversation as resolved.
Show resolved
Hide resolved
|
||
from cuxfilter.dashboard import DashBoard | ||
from ._version import get_versions | ||
|
||
__version__ = get_versions()["version"] | ||
del get_versions | ||
__version__ = "23.08.00" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You'll need to borrow some logic from here for dask-cuda: https://github.com/rapidsai/cudf/blob/1899fb1990eae693bd74f92fa4a3f6a217889210/.github/workflows/pr.yaml#L132-L133
dask-cuda has to install from source, since the
23.8.*a*
packages on PyPI are pre-release / alphas. We can't really usepip install ... --pre
because it uses pre-release versions of all packages.Also note that you'll need to change
update-version.sh
to accomodate this. https://github.com/rapidsai/cudf/blob/1899fb1990eae693bd74f92fa4a3f6a217889210/ci/release/update-version.sh#L112I'm not sure if you need just dask-cuda or also dask/distributed so I have suggestions for both.
All of dask/distributed/dask-cuda:
Only dask-cuda:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah okay got it. It would just be dask-cuda at this point. Thanks again!
@bdice I am assuming this is a temporary fix, until dask-cuda has a PyPI stable release (23.8.*?), right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually @bdice, we don't need
dask_cuda
as a direct dependency, since it's only used in the dask_cudf based notebooks. So I removed the direct dependency, and added it tonotebooks
dependency onlyThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great that the dependency could be moved.
To answer your question about whether this would be a temporary fix, it isn't temporary. The dask-cuda wheels used in CI are always pre-release alphas. We are always testing RAPIDS packages on
branch-XX.YY
against pre-releases of dask-cudaXX.YY
to ensure that when the releases are made, everything works together as intended.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
got it, good to know. Thanks again for the thorough review. Let me know if this looks good to merge.