[FEA]: Update available cmake targets

[jrhemstad]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this request and that I agree to the Code of Conduct

Area

Infrastructure

Is your feature request related to a problem? Please describe.

As a user of CCCL + cmake, I would like to have a convenient cmake target to link against that makes all of the CCCL code available on my include path as well as set any potential compiler flags.

Describe the solution you'd like

I would like a cccl::cccl target that I can link against that provides Thrust/CUB/libcu++ to my application.

target_link_libraries(my_app PRIVATE cccl::cccl)

This should effectively provide the same experience as if I were using CCCL packaged with the CTK, meaning I can #include anything from <thrust/...> <cub/...> <cuda/...>. Furthermore, the thrust::device backend should be CUDA and the thrust::host backend should be cpp. There is no expectation that using any of the other should be available.

I would also like individual targets for Thrust/CUB/libcu++ exposed under the cccl:: namespace, e.g., cccl::thrust, cccl::cub, cccl::libcudacxx.

In order to optimize to make the common case easy and convenient, I'd like to not have to run any additional cmake functions to expose the targets other than add_subdirectory (or whatever else is used by things like CPM).

Describe alternatives you've considered

No response

Additional context

For CUB and libcu++, I expect this will not be an issue, but this will be a departure from the status quo for Thrust where you currently have to invoke a cmake function in order to select which execution backends you want before the link targets are exposed.

I believe the overwhelmingly common case is that people use the CUDA backend for thrust::device and sequential cpp backend for thrust::host. We should optimize to make this common case easy and convenient with the cccl::cccl target.

We can still require invoking a cmake function in order to expose more customized targets where people wish to configure alternative backends for thrust::device/host.

[alliepiper]

My biggest concern is the usecase of "I use Thrust with a CPU parallel backend and don't want to enable the CUDA language or Thrust CUDA backend at all."

If we always provide a cccl::cccl/cccl::thrust target that defaults to CPP/CUDA host/device, we must enable CUDA and the Thrust CUDA backend, which can be expensive and problematic:

1. What if the current environment doesn't have the CTK installed? Including CCCL will cause CMake to try to enable CUDA and search for a CUDA compiler, then fail when it's not found.
2. On windows especially, enabling the CUDA language in CMake adds significant overhead to the configure step.

My initial thought to work around this is to still provide the subproject packages, so a user can just pull in Thrust (add_package(Thrust) / add_subdirectory(cccl/thrust) without any configured backends and selectively enable the ones they want. But if they include CCCL, I think it's a reasonable expectation that CTK will be present and desired, so a cccl::thrust target can be configured as described above.

[alliepiper]

The other concern is how we spell the target names. We're in a bit of a mess right now with capitalization conventions:

libcudacxx::libcudacxx vs. CUB::CUB vs. Thrust::Thrust

Introducing cccl::libcudacxx is fine, but cccl::cub and cccl::thrust would be different spellings than the native packages, which is somewhat ugly. Do we:

a. Keep the existing capitalization for the cccl:: targets (cccl::libcudacxx, cccl::Thrust, cccl::CUB)
b. Use lowercase for cccl:: targets and change the packages to also use all lowercase (I do not like this, unnecessarily disruptive for users)
c. Have mismatched capitalization between the cccl targets and the subproject targets (I do not like this, unnecessarily inconsistent)
d. Use lowercase targets for cccl:: and provide lowercase targets in subproject while also keeping the mixed/upper case targets around as aliases, eventually perhaps deprecating them down the road (nice compromise, but feels hacky).

[miscco]

I am seconding Allison that we should provide support for users to easily include individual projects on their own.

That said, I imagine the more common use case to be an include cccl::something

Given that we are introducing new targets I would greatly appreciate, if we could just settle on the common cmake practice and follow that for all repos. I am saying that not even knowing what the common cmake practice is. But having three different spellings is just crazy

[jrhemstad]

If we always provide a cccl::cccl/cccl::thrust target that defaults to CPP/CUDA host/device, we must enable CUDA and the Thrust CUDA backend, which can be expensive and problematic:

Hm, yeah, that's more complicated than I originally thought about.

So what if instead of requiring invoking a cmake function to populate the targets, we make them a cmake option like:

option(CCCL_ENABLE_CUDA "Enable CUDA targets" ON)
option(CCCL_ENABLE_THRUST_OPENMP_BACKEND "Enable the TBB backend for Thrust" OFF)
option(CCCL_ENABLE_THRUST_TBB_BACKEND "Enable the OpenMP backend for Thrust" OFF)

So then if I'm in a non-CUDA context, then I can do:

CPMAddPackage(
    NAME CCCL
    GITHUB_REPOSITORY nvidia/cccl
    OPTIONS "CCCL_ENABLE_CUDA OFF" "CCCL_ENABLE_THRUST_OPENMP_BACKEND ON"
)

target_link_library(my_app cccl::cccl) # This won't link against the cccl::CUB target, nor will it require the CUDA language 

a. Keep the existing capitalization for the cccl:: targets (cccl::libcudacxx, cccl::Thrust, cccl::CUB)

Yeah, do this. I didn't intentionally change the capitalization, I just wasn't sure what it was to begin with and I don't have a dog in this race 🙂.

I am seconding Allison that we should provide support for users to easily include individual projects on their own.

That was always the intention. I'm trying to provide a target that makes using everything easier.

[alliepiper]

So what if instead of requiring invoking a cmake function to populate the targets, we make them a cmake option like:

What about CCCL_THRUST_DEFAULT_HOST_BACKEND and CCCL_THRUST_DEFAULT_DEVICE_BACKEND variables that can be set to CPP, OMP, TBB, or CUDA to control the configuration of the default cccl::Thrust target?

Or, if the user would rather handle Thrust configuration without creating a cccl::Thrust target, they could always just do something like:

find_package(cccl COMPONENTS libcudacxx CUB) # cccl::cccl only provides libcu++ and CUB
find_package(Thrust) # Find thrust separately
thrust_configure_target(MyThrust ...) # Create a custom Thrust target
target_link_libraries(cccl::cccl INTERFACE MyThrust) # Add the custom Thrust target to cccl::cccl

[jrhemstad]

What about CCCL_THRUST_DEFAULT_HOST_BACKEND and CCCL_THRUST_DEFAULT_DEVICE_BACKEND variables that can be set to CPP, OMP, TBB, or CUDA to control the configuration of the default cccl::Thrust target?

I like this even better than what I said.

Would it still make sense to have a CCCL_ENABLE_CUDA option (or something similar)?

I'm thinking of the use case similar to Thrust where someone may want to use libcu++ in a purely host-only environment where CUDA isn't available. How would we enable them to do that?

[alliepiper]

Would it still make sense to have a CCCL_ENABLE_CUDA option (or something similar)?

The libcu++ package is a lot more lightweight than Thrust, and it shouldn't have any issues -- we just generate a single libcudacxx::libcudacxx interface target with the include directory set.

Thrust is the only package that has any real complexity here, since libcudacxx and CUB do not have optional dependencies, so as long as the Thrust usecase is covered, we should be fine, I don't think we'll need any extra options for those.

[alliepiper]

I am saying that not even knowing what the common cmake practice is. But having three different spellings is just crazy.

I believe that the CMake convention is to use the common project spelling. Looking at how the projects are spelled in the READMEs, etc, libcudacxx, Thrust, and CUB are how the projects are commonly called, so we used those spellings.

[alliepiper]

Which reminds me -- the CCCL project is named CCCL and discovered via find_package(CCCL), so we should name these targets CCCL::CCCL, CCCL::libcudacxx, CCCL::Thrust, and CCCL::CUB.

[jrhemstad]

I don't think we'll need any extra options for those

Aren't libcu++ and CUB set up to enable the CUDA language such that if you attempt to use them in a context where CUDA isn't available then cmake will fail?

[alliepiper]

Aren't libcu++ and CUB set up to enable the CUDA language such that if you attempt to use them in a context where CUDA isn't available then cmake will fail?

Nope -- they're both simple interface targets that only provide include directories. Thrust is much more complicated and has to search for dependencies depending on the backends, so that's the only project we need to be careful with.

[robertmaynard]

I would prefer we investigate using find_package(CCCL COMPONENTS) to control thrust host/device selection.

• find_package(CCCL) -> Default to CUDA device backend, openmp && tbb disabled
• find_package(CCCL COMPONENTS THRUST_TBB) -> Use TBB as the device backend

If users need a more complex setup for the host backend, they can use the Thrust functions to do that.

[jrhemstad]

I would prefer we investigate using find_package(CCCL COMPONENTS) to control thrust host/device selection.

How do you do that through CPM?

[robertmaynard]

I would prefer we investigate using find_package(CCCL COMPONENTS) to control thrust host/device selection.

How do you do that through CPM?

CPMFindPackage has support for components via the FIND_ARGS parameter. Since CPMAddPackage doesn't use find_package at all this would have no impact in that workflow.

Depending on the logic the CCL-config could map from the COMPONENTS syntax to the proposed variables ( e.g CCCL_ENABLE_CUDA ). That would provide a nice interface to users that use find_package and those that want to use CCCL directly from source.

[alliepiper]

I would prefer we investigate using find_package(CCCL COMPONENTS) to control thrust host/device selection.

I have thought about this approach as well. We do use find_package(CCCL COMPONENTS Thrust CUB libcudacxx) to select subprojects, and we also support find_package(Thrust COMPONENTS TBB OMP CPP CUDA) to pre-load the backends. However, this Thrust component syntax still requires using the thrust_configure_target(....) to build a target with the backends mapped to host/device.

To me, find_package(CCCL COMPONENTS Thrust_TBB) is ambiguous -- is this meant to be host=cpp device=tbb, or host=tbb device=cuda? It'd also be a bit confusing because assuming either would be confusingly different from how the Thrust package works when used directly. We could introduce components like Thrust_CPP_TBB but I think we can do better.

Thrust also supports this feature that we could use for CCCL: https://github.com/NVIDIA/cccl/blob/main/thrust/thrust/cmake/README.md#configure-target-from-cache-options

# In CCCL package
find_package(Thrust)
# The CCCL::Thrust target will be configured from advanced cache options:
thrust_configure_target(CCCL::Thrust FROM_OPTIONS
  HOST_OPTION CCCL_THRUST_HOST_SYSTEM
  DEVICE_OPTION CCCL_THRUST_DEVICE_SYSTEM
  ADVANCED
)

# User code:
set(CCCL_THRUST_HOST_SYSTEM TBB)
set(CCCL_THRUST_DEVICE_SYSTEM CUDA)
find_package(CCCL)
target_link_libraries(foodledoo PRIVATE CCCL::Thrust)