-
Notifications
You must be signed in to change notification settings - Fork 511
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[rapids] removed spark tests, updated to a more recent rapids release #1219
base: master
Are you sure you want to change the base?
Conversation
I prefer this to #1218 |
/gcbrun |
/gcbrun |
2 similar comments
/gcbrun |
/gcbrun |
Should we increase the machine type from n1-standard-4 to n1-standard-16 |
cuda11 has been manually tested with all versions. |
/gcbrun |
1 similar comment
/gcbrun |
tests are failing for
|
/gcbrun |
1 similar comment
/gcbrun |
[edit: this was a misconfiguration in the systemd unit] It looks like the dask infrastructure is out of date and I'll have to target 2023.12 instead.
|
I also need to reduce the python abi to 3.10 |
/gcbrun |
6 similar comments
/gcbrun |
/gcbrun |
/gcbrun |
/gcbrun |
/gcbrun |
/gcbrun |
/gcbrun |
/gcbrun |
/gcbrun |
…and rapids init actions
/gcbrun |
/gcbrun |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your continued persistence here CJ! 🙏
At this point, would it be worthwhile to try with RAPIDS 24.10?
rapids/rapids.sh
Outdated
function is_cuda12() { [[ "${CUDA_VERSION%%.*}" == "12" ]] ; } | ||
function is_cuda11() { [[ "${CUDA_VERSION%%.*}" == "11" ]] ; } | ||
|
||
readonly DEFAULT_DASK_RAPIDS_VERSION="24.08" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
readonly DEFAULT_DASK_RAPIDS_VERSION="24.08" | |
readonly DEFAULT_DASK_RAPIDS_VERSION="24.10" |
rapids/rapids.sh
Outdated
# SPARK config | ||
readonly DEFAULT_SPARK_RAPIDS_VERSION="24.08.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
readonly DEFAULT_SPARK_RAPIDS_VERSION="24.08.0" | |
readonly DEFAULT_SPARK_RAPIDS_VERSION="24.10.0" |
/gcbrun |
I'm having a hard time with 24.08 right now due to custreamz not matching:
|
Is Dask installed before RAPIDS is installed? If so, could they be combined into the same install step? Think that would minimize conflicts Should add RAPIDS 24.10 was recently released so is using a newer version of Dask, which also may minimize conflicts |
okay, I have a lot of changes to merge from some work I did while I was adding secure-boot support to the custom-images repo. It's kind of a lot. |
rapids/BUILD * removed dependence on verify_xgboost_spark.scala - this belongs in [spark-rapids] * removed dependence on dask rapids/rapids.sh * added utility functions * reverted dask_spec="dask>=2024.5" * using realpath to /opt/conda/miniconda3/bin/mamba instead of default symlink * remove conda environment [dask] if installed * asserting existence of directory depended on by the script when run as custom-images script * created exit_handler and prepare_to_install functions to set up and clean up rapids/test_rapids.py * refactored to make use of systemd unit defined in rapids.sh * added retry to ssh * removed condition to keep tests from running on 2.0 images
/gcbrun |
Tested with CUDA=11 and CUDA=12