Standalone CMake build

[srajama1]

The current CMake build system works only with Trilinos and we use a script to compile standalone. Need to fix the CMake system.

[ndellingwood]

@william76 - adding you to the issue, in case you have time to spare on this.

[jjwilke]

We will want to use a:

target_link_libraries(kokkoskernels kokkos)

Best practice is to use a namespace for imported targets so that cmake knows it's a CMake target and not just a library. AFAIK kokkos does not namespace targets currently.

target_link_library(kokkoskernels Kokkos::kokkos)

Otherwise you can end up with just a -lkokkos appended and hard-to-debug errors.
I will proceed without namespaces - but am curious if this is something to consider.

[jjwilke]

Related to kokkos/kokkos#674 and kokkos/kokkos#1924

[jjwilke]

TL; DR: KokkosKernels needs a modest overhaul to the build system to do modern CMake correctly. Kokkos itself might need a couple fixes, too.

Okay - I think I need clarification. This is my impression of the current build state of both kokkos and kokkoskernels (which may be incorrect).

• Kokkos kernels does NOT target_link_libraries(kokkoskernels kokkos) in either a standalone or Trilinos build
• Kokkos itself does not have a target_include_directories and as such does not have:

target_include_directories(kokkos INTERFACE
  $<BUILD_INTERFACE:src>
  $<INSTALL_INTERFACE:include>
)

Even if we link kokkoskernels against a kokkos CMake target, kokkos is not currently propagating include targets.

• Kokkos IS using target_compile_options so that OpenMP and -std=c++1X flags are propagating correctly when linking against kokkos CMake target.
• Kokkos kernels itself is not using target_include and is instead using include_directories. All of these need to be changed if a kokkoskernels CMake target is going to correctly propagate include paths (particularly kokkos transitive includes)
• Kokkos actually exports the build tree to .cmake/packages/Kokkos by default. Thus any Kokkos build (not install) ends up in the CMake package repository. Any library calling find_package(kokkos) will find a random build tree (not install). make install does not export to the CMake package repository.

[crtrott]

@jjwilke: please feel free to consider everything you want to consider and fix the above mentioned issues in both Kokkos and KokkosKernels ;-)

[jjwilke]

Feature branch of Kokkos at jjwilke/kokkos.git@4a50aed548c187f0365f624fab34084c1f1521f2
contains the infrastructure needed to support this for CMake >=3.0.

[srajama1]

Is this ready to come in as a PR ?

[ndellingwood]

@srajama1 @jjwilke I think we should test pretty extensively with Trilinos first to make sure there was no accidental compatibility breakage, does that sound reasonable?

[crtrott]

@ndellingwood this sounds very very reasonable. We also need to talk to the LANL folks who rely on current Kokkos cmake support that the Kokkos core changes didn't break there stuff (and/or they like the changes necessary).

[jjwilke]

Yes, yes. Much more testing needs to be done. I'm particularly worried about case-sensitive file names since Darwin mucks this up by not being case-sensitive.

@ndellingwood: In theory, nothing for Trilinos was broken - but obviously this should be tested. As far as I can tell, the kokkos_build.cmake file is only included when not building with Trilinos. This is the only file I modified in a significant way. Everything else uses TRIBITS_X(...)

CMake 2.8 - which is the minimum for Trilinis (I think) - supports target_include_directories and target_link_libraries, but does not do transitive dependencies. The most likely point of breakage is the examples and tests relying on transitive includes/libs which will work in the Kokkos minimum 3.3 but will not work in the Trilinos minimum (2.8).

[crtrott]

Trilinos is actually at 3.10 as a minimum for CMake.

[jjwilke]

The first draft is at kokkos-kernels on the kokkos repository kokkos

TODO: package configure file, if we want to support non-cmake projects

Transitive dependencies appear to be working correctly for standalone builds using:

find_package(KokkosKernels)
add_executable(myexe ...)
target_link_libraries(myexe kokkos::kokkoskernels)

Kokkos transitive dependencies are propagating correctly when linking.
Kokkos kernels used a lot more Tribits. We will want to do a lot more testing inside of Trilinos and with other configurations.

A minor complication here (and requires testing with more configurations) - Kokkos kernels by default with no ETI does not appear to add any sources to the list? This means kokkoskernels must be declared as an interface library otherwise this generates a CMake error.

[srajama1]

It is true we do not do ETI on some of our kernels. However, everything in src/impl/generated_specializations_cpp should get into a generated library based on what ETI options are enabled right ?

[jjwilke]

That is my understanding - but then KokkosKernels becomes a header-only library? Is that correct?

I believe the "correct" way to do a header-only library going forward is to declare it as an interface, but I will try to confirm this on the CMake list.

[srajama1]

I am missing something. Do you mean to say when ETI is off we are a header only library ? We can add an empty dummy.cpp to workaround it.

[jjwilke]

CMake makes header-only libraries fairly easy. There was a type-o (fixed locally) which was causing sources not to get added. The behavior was supposed to be ETI on by default, which builds a conventional library. I was surprised to not see any source files getting built by default (hence the question), but that issue is fixed.

[jjwilke]

I am changing all executables generated to NOEXEPREFIX so that they have the same names regardless of the parent project. If there is a reason this would break anything, please let me know.

[srajama1]

The old scripts won't work, but that is fine.

[jjwilke]

Tests/platforms currently passing:

• Mac OS, Standalone, OpenMP, Serial, Pthread: test_all_sandia with CMake
• Mac OS, Trilinos, OpenMP, Serial with Tpetra
• Mutrino, Standalone, OpenMP, Serial: test_all_sandia with CMake
• Mutrino, Trilinos, OpenMP, Serial with Tpetra
  Tests/platforms currently failing:

[jjwilke]

Let's call this done, eh?