-
Notifications
You must be signed in to change notification settings - Fork 511
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixing rapids test for presubmits #1218
Conversation
/gcbrun |
rapids/test_rapids.py
Outdated
@@ -53,33 +53,6 @@ def verify_spark_job(self): | |||
self.remove_test_script(self.XGBOOST_SPARK_TEST_SCRIPT_FILE_NAME, | |||
instance_name) | |||
|
|||
@parameterized.parameters(("STANDARD", ["m", "w-0"], GPU_T4, None), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what? no. We need to make the test pass, not delete the test. It's working for me. It takes 50+ minutes for cuda version 12. Have I not created a PR yet?
No, I have not. Let me push. We'll want to abandon this for #1219 I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It got pushed by mistake. This change was for my local testing.
/gcbrun |
dependent on #1212 |
@@ -131,7 +132,7 @@ function install_dask_rapids() { | |||
fi | |||
# Install RAPIDS, cudatoolkit | |||
mamba install -m -n 'dask-rapids' -y --no-channel-priority -c 'conda-forge' -c 'nvidia' -c 'rapidsai' \ | |||
"cudatoolkit=${CUDA_VERSION}" "pandas<1.5" "rapids=${RAPIDS_VERSION}" "python=${python_ver}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no, we need to allow customer to select by rapids version. This needs to stay.
@@ -131,7 +132,7 @@ function install_dask_rapids() { | |||
fi | |||
# Install RAPIDS, cudatoolkit | |||
mamba install -m -n 'dask-rapids' -y --no-channel-priority -c 'conda-forge' -c 'nvidia' -c 'rapidsai' \ | |||
"cudatoolkit=${CUDA_VERSION}" "pandas<1.5" "rapids=${RAPIDS_VERSION}" "python=${python_ver}" | |||
"cudatoolkit=${CUDA_VERSION%%.*}" "pandas<1.5" "rapids" "python=${python_ver}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
selecting by major version is right.
I think we should remove the standalone dask action and move all dask installation instructions into this one. spark-rapids is supposed to be the way to install rapids + spark. We can consider this one dask-rapids but named rapids. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to exercise CUDF and xgboost to confirm that the installation completed successfully
@@ -1,39 +1,35 @@ | |||
import cudf |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cuda accelerated data frames is an important component. I don't think we want to disable tests for it.
|
||
from dask.distributed import Client |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an important component of the test.
|
||
def test_rapids(): | ||
# confirm RAPIDS and xgboost are available | ||
df = cudf.DataFrame() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do want to confirm that rapids and xgboost are available
Closing this PR as @cjac is taking care of rapids and gpu related init actions. |
No description provided.