GPU support for CI

[charlesbluca]

It would be nice if we were able to optionally run a subset of the test suite on a machine/container with GPU support. This could allow us to preemptively catch breaking changes such as the one that caused rapidsai/dask-cuda#634, which initially went under the radar because it broke something in the diagnostics.nvml realm.

There are a lot of options for this:

• @quasiben suggested checking out Cirun.io, which would allow us to use a VM in the cloud as a self-hosted runner through GHA; pystatgen/sgkit#567 is an example of adding that support to a repo (although we'd probably also need to make/manage an account of some sort)
• There should be a way to trigger tests in a cloud VM without Cirun.io, though it might be a little more complicated and I'm not sure if we can run the tests through GHA
• RAPIDS has gpuCI for the bulk of its GPU testing, though that would probably require a great deal of ongoing collaboration to keep working

[jrbourbeau]

Yeah totally agree GPU testing would be great to have. There's an existing dask/community issue (xref dask/community#138) where this was discussed previously. Is it okay if we close this issue in favor of continuing the discussion over there?

[charlesbluca]

Sure, thanks for the redirect!