updates to external tests (#547)

This PR resolves the external tests still failing: https://github.com/rapidsai/cuxfilter/actions/runs/6485671242/job/17612261430.

Basically separately installing direct dependencies and then installing indirect dependencies from dependencies.yaml was downgrading rapids versions to 23.08 separately. This PR updates the dependencies to be in a single place (external_dependencies.yaml), while also adding it to `ci/release/update-version.sh`

There were a couple of dask.array tests in datashader which were failing when using `pytest`  with `--numprocesses` flag, which is resolved here by separately running that test without the flag.

cc @exactlyallan

Authors:
  - Ajay Thorve (https://github.com/AjayThorve)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)

URL: #547

EXTERNAL_TESTS.md

@@ -0,0 +1,57 @@
+# External Tests Workflow
+
+This document provides an overview of the GitHub Actions workflow (`.github/workflows/test-external.yaml`) and associated script (`ci/test_external.sh`) for running external tests on specified Python libraries, such as Datashader and Holoviews.
+
+## Purpose
+
+The purpose of this workflow is to perform GPU testing on external party dependencies. It involes the following steps:
+
+1. Create a Conda environment named `test_external`.
+2. Install external dependencies specified in `ci/utils/external_dependencies.yaml`.
+3. Clone specified Python libraries from their respective GitHub repositories.
+4. Install test dependencies for each library.
+5. Run GPU tests on the specified libraries using Pytest.
+
+## Workflow Configuration
+
+### Workflow Trigger
+
+The workflow is triggered in two ways:
+
+1. **Manual Trigger:** You can manually trigger the workflow by selecting the "GPU testing for external party dependencies" workflow and providing the following inputs:
+
+   - `external-project`: Specify the project to test (`datashader`, `holoviews`, or `all`).
+   - `pr_number`: (Optional) If testing a pull request, provide the PR number.
+
+2. **Scheduled Trigger:** The workflow runs automatically every Sunday evening (Pacific Time) using a cron schedule (`0 0 * * 1`).
+
+## Script (`test_external.sh`)
+
+The script is responsible for setting up the Conda environment, installing dependencies, cloning specified Python libraries, and running GPU tests. Key steps in the script include:
+
+1. **Create Conda Environment:** Creates a Conda environment named `test_external` and installs external dependencies from `external_dependencies.yaml`.
+
+2. **Clone Repositories:** Clones GitHub repositories of specified Python libraries (`datashader`, `holoviews`, or both).
+
+3. **Install Dependencies:** Installs test dependencies for each library using `python -m pip install -e .[tests]`.
+
+4. **Run Tests:** Gathers GPU tests containing the keywords `cudf` and runs them using Pytest. The number of processes is set to 8 by default, but specific tests (`test_quadmesh.py`) are run separately.
+
+## Running External Tests
+
+To manually trigger the workflow and run external tests:
+
+1. Navigate to the "Actions" tab in your GitHub repository.
+2. Select "GPU testing for external party dependencies" workflow.
+3. Click the "Run workflow" button.
+4. Provide inputs for `external-project` and `pr_number` if needed.
+
+## Contributing
+
+Contributors can use this workflow to test changes in external libraries on the RAPIDS AI ecosystem. When contributing, follow these steps:
+
+1. Make changes to the external library code.
+2. Push the changes to your fork or branch.
+3. Trigger the workflow manually by selecting the appropriate inputs.
+
+For additional information, refer to the [GitHub Actions documentation](https://docs.github.com/en/actions).

ci/release/update-version.sh

@@ -49,7 +49,7 @@ DEPENDENCIES=(
   cugraph
   cuspatial
 )
-for FILE in dependencies.yaml conda/environments/*.yaml; do
+for FILE in dependencies.yaml conda/environments/*.yaml ci/utils/external_dependencies.yaml; do
   for DEP in "${DEPENDENCIES[@]}"; do
     sed_runner "/-.* ${DEP}==/ s/==.*/==${NEXT_SHORT_TAG_PEP440}.*/g" ${FILE};
   done

ci/test_external.sh

@@ -6,21 +6,8 @@ set -e
 rapids-logger "Create test_external conda environment"
 . /opt/conda/etc/profile.d/conda.sh
 
-RAPIDS_VERSION=23.12.*
-
-rapids-mamba-retry create \
-    -n test_external \
-    --override-channels \
-    -c rapidsai-nightly \
-    -c nvidia \
-    -c conda-forge \
-    cuxfilter="${RAPIDS_VERSION}" cudf="${RAPIDS_VERSION}" dask-cudf="${RAPIDS_VERSION}" \
-    python="${RAPIDS_PY_VERSION}" cuda-version="12.0" cupy="12.0"
-
 # Install external dependencies into test_external conda environment
-pushd ./ci/utils
-rapids-mamba-retry env update -n test_external -f external_dependencies.yaml
-popd
+rapids-mamba-retry env update -f ./ci/utils/external_dependencies.yaml
 
 conda activate test_external
 
@@ -78,8 +65,8 @@ if [ "$PROJECT" = "all" ]; then
     do
         rapids-logger "gathering GPU tests for $LIBRARY"
         TEST_DIR="$LIBRARY/$LIBRARY/tests"
-        # Find all Python scripts containing the keywords cudf or dask_cudf
-        FILES+=" $(grep -l -R -e 'cudf' --include='*.py' "$TEST_DIR")"
+        # Find all Python scripts containing the keywords cudf or dask_cudf except test_quadmesh.py
+        FILES+=" $(grep -l -R -e 'cudf' --include='*.py' "$TEST_DIR" | grep -v test_quadmesh.py)"
     done
 else
     rapids-logger "gathering GPU tests for $PROJECT"
@@ -95,5 +82,11 @@ set +e
 rapids-logger "running all gathered tests"
 DATASHADER_TEST_GPU=1 pytest --numprocesses=8 $FILES
 
+if [[ "$PROJECT" = "all" ]] || [[ "$PROJECT" = "datashader" ]]; then
+    # run test_quadmesh.py separately as dask.array tests fail with numprocesses
+    rapids-logger "running test_quadmesh.py"
+    DATASHADER_TEST_GPU=1 pytest datashader/datashader/tests/test_quadmesh.py
+fi
+
 rapids-logger "Test script exiting with value: $EXITCODE"
 exit ${EXITCODE}

ci/utils/external_dependencies.yaml

@@ -1,7 +1,14 @@
 name: test_external
 channels:
+  - rapidsai-nightly
   - conda-forge
+  - nvidia
 dependencies:
+  - cudf==23.12.*
+  - dask-cudf==23.12.*
+  - cuxfilter==23.12.*
+  - cuda-version=12.0
+  - python=3.10
   - tensorflow
   - xarray-spatial
   - pycaret