[FEA] Replace defaulted stream value in libcudf APIs with cudf::cuda_default_stream

[jrhemstad]

Is your feature request related to a problem? Please describe.

libcudf public APIs do not current expose CUDA stream arguments (#925).

However, every public API internally invokes a corresponding detail:: API that has a rmm::cuda_stream_view parameter defaulted to rmm::cuda_stream_default.

I would like to be able to globally control this default value at the scope of libcudf (meaning, not changing it at the RMM level which could be much broader).

Describe the solution you'd like

Add a global cudf::cuda_default_stream much like the rmm::cuda_default_stream
and replace every use of rmm::cuda:default_stream with cudf::cuda_default_stream.

Furthermore, I would like the value of this cudf::cuda_default_stream to depend on the CUDF_USE_PER_THREAD_DEFAULT_STREAM option.

namespace cudf{
#if define(CUDF_USE_PER_THREAD_DEFAULT_STREAM)
   static constexpr rmm::cuda_stream_view cuda_default_stream{rmm::cuda_stream_per_thread};
#else
   static constexpr rmm::cuda_stream_view cuda_default_stream{};
#endif
}

Additional Context

I explicitly made the configuration of cuda_default_stream's value depend on the cudf specific CUDF_USE_PER_THREAD_DEFAULT_STREAM option instead of the CUDA_API_PER_THREAD_DEFAULT_STREAMoption even though the former implies the latter.

This is to minimize potential astonishment. If someone were to compile libcudf manually using --default-stream per-thread they would not expect the behavior described above. The behavior above requires opting in explicitly through the cudf specific option.

[jbrennan333]

In terms of the build process, would this still be keyed off of PER_THREAD_DEFAULT_STREAM?
That is, will we define both CUDA_API_PER_THREAD_DEFAULT_STREAM and CUDF_USE_PER_THREAD_DEFAULT_STREAM when PER_THREAD_DEFAULT_STREAM=ON?

[jrhemstad]

In terms of the build process, would this still be keyed off of PER_THREAD_DEFAULT_STREAM? That is, will we define both CUDA_API_PER_THREAD_DEFAULT_STREAM and CUDF_USE_PER_THREAD_DEFAULT_STREAM when PER_THREAD_DEFAULT_STREAM=ON?

See #10862

TL;DR

CUDF_USE_PER_THREAD_DEFAULT_STREAM replaces PER_THREAD_DEFAULT_STREAM and implies CUDA_API_PER_THREAD_DEFAULT_STREAM.

[jbrennan333]

Clarification: The rmm name is rmm::cuda_stream_default. I'm assuming we want the new default to be cudf::cuda_stream_default, to match the rmm value, not cudf::cuda_default_stream?

[jrhemstad]

Clarification: The rmm name is rmm::cuda_stream_default. I'm assuming we want the new default to be cudf::cuda_stream_default, to match the rmm value, not cudf::cuda_default_stream?

I was actually considering an alternative that would perhaps be less ambiguous.

"default stream" is ambiguous because it could mean "The CUDA default stream" or "the default value of the stream parameter in every function". Really, we want the latter. It's a subtle and pedantic difference, but one I think that is important.

Therefore, something like cudf::default_stream_value appeals to me for being more explicit about what it means. Or even cudf::defaulted_stream_value.

[jbrennan333]

Thanks @jrhemstad!

[jbrennan333]

I put up PR #10877 for this. It only replaces rmm::cuda_stream_default in the cases that impact nvcomp.
Will need to follow up to change the rest.

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[jbrennan333]

This issue is still in progress. The remaining work is covered by #11082.