Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KokkosKernels: Don't list include for non-existant 'batched' build dir (trilinos/Trilinos#11966) #1867

Merged

Conversation

bartlettroscoe
Copy link
Contributor

This matches the commit bartlettroscoe@aee776e in Trilinos PR:

This is a clear defect that just happens to not cause a problem unless you try to use the KokkosKernelsTargets.cmake file generated for the build dir (see trilinos/Trilinos#11966 (comment)).

See the log message and embedded comment for more details.

trilinos/Trilinos#11966)

See the embedded comment.  This fixes the generated KokkosKernelsTargets.cmake
file in the build dir when tests are not enabled.

NOTE: This is a very non-conventional CMake build system.  Your never see
CMakeLists.txt files that are included with include().  These should always be
pulled in using add_subdirectory().  If you want to just include() some CMake
code, you call it `<SomeLogicalBaseName>.cmake` and then include() it.
@kokkos-devops-admin
Copy link

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

Copy link
Contributor

@ndellingwood ndellingwood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bartlettroscoe !

@ndellingwood ndellingwood added CompatibilityUpdate TrilinosPatchMatch Apply this label for PR's mirroring changes submitted directly to Trilinos labels Jun 14, 2023
@ndellingwood
Copy link
Contributor

This will be cherry-picked to the release-candidate-4.1.00 branch

@ndellingwood ndellingwood added the AT: PRE-TEST INSPECTED Mark this PR as approved for testing. label Jun 14, 2023
@kokkos-devops-admin kokkos-devops-admin removed the AT: PRE-TEST INSPECTED Mark this PR as approved for testing. label Jun 14, 2023
@kokkos-devops-admin
Copy link

Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; this inspection will remain valid until a new commit to source branch is performed.

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight

  • Build Num: 692
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10

  • Build Num: 283
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC1020

  • Build Num: 306
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC1020_Light_LayoutRight

  • Build Num: 576
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_GCC1020

  • Build Num: 537
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_INTEL19

  • Build Num: 626
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_CLANG1001

  • Build Num: 679
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_A64FX_Tpls_ARMPL2110

  • Build Num: 480
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_A64FX_GCC1020

  • Build Num: 475
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_VEGA908_ROCM520

  • Build Num: 475
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Using Repos:

Repo: KOKKOSKERNELS (bartlettroscoe/kokkos-kernels)
  • Branch: tril-11966-bad-batched-incl-dir
  • SHA: aee776e
  • Mode: TEST_REPO

Pull Request Author: bartlettroscoe

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight

  • Build Num: 692
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10

  • Build Num: 283
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC1020

  • Build Num: 306
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC1020_Light_LayoutRight

  • Build Num: 576
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_GCC1020

  • Build Num: 537
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_INTEL19

  • Build Num: 626
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_CLANG1001

  • Build Num: 679
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_A64FX_Tpls_ARMPL2110

  • Build Num: 480
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_A64FX_GCC1020

  • Build Num: 475
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_VEGA908_ROCM520

  • Build Num: 475
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/bartlettroscoe/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA aee776e
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA af312b9
PR_LABELS CompatibilityUpdate;TrilinosPatchMatch
PULLREQUESTNUM 1867
TEST_REPO_ALIAS KOKKOSKERNELS

@kokkos-devops-admin
Copy link

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ ndellingwood ]!

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge

KOKKOSKERNELS_INCLUDE_DIRECTORIES(REQUIRED_DURING_INSTALLATION_TESTING ${CMAKE_CURRENT_SOURCE_DIR}/batched)
IF(KokkosKernels_ENABLE_TESTS OR KokkosKernels_ENABLE_TESTS_AND_PERFSUITE)
# Adding unit-tests
KOKKOSKERNELS_INCLUDE_DIRECTORIES(${CMAKE_CURRENT_BINARY_DIR}/batched)
Copy link
Contributor

@e10harvey e10harvey Jun 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I apologize in advance for any misunderstanding on my part. Given that we append the correct include directories to KK_INCLUDE_DIRS above these lines and have this for-each loop in our top-level CMake:

  FOREACH(DIR ${KK_INCLUDE_DIRS})
    KOKKOSKERNELS_INCLUDE_DIRECTORIES(${DIR})
    APPEND_GLOB(HEADERS ${DIR}/*.hpp)
  ENDFOREACH()

I do not see a need to invoke KOKKOSKERNELS_INCLUDE_DIRECTORIES here.

Where are these includes required in the KokkosKernels build system? Does Trilinos build without them?

Why isn't this same change needed in blas/CMakeLists.txt?

Why don't we have similar KOKKOSKERNELS_INCLUDE_DIRECTORIES invocations with REQUIRED_DURING_INSTALLATION_TESTING in sparse/CMakeLists.txt?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really these include directories should be added in the CMakeLists.txt files for the tests themselves and NOT added to the exported include directories for the package that show up in the KokkosKernelsConfig.cmake file. Also, someone should consider refactoring this build system to not include CMakeLists.txt files. I have never seen that before and that is confusing. A CMakeLists.txt file is meant to be brought into a project using add_subdirectory().

I think some straightforward refactorings of this build system like the proper usage of CMakeLists.txt files might make it more standard and easier to understand and maintain.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That may be true but I am still at a loss for where these include directories are currently required. No where in the code base do I see #include "dense/..." or #include "sparse/...". I also deleted these two KOKKOSKERNELS_INCLUDE_DIRECTORIES calls and found that the KK code base configures and builds without errors. Is KOKKOSKERNELS_INCLUDE_DIRECTORIES(REQUIRED_DURING_INSTALLATION_TESTING ${CMAKE_CURRENT_SOURCE_DIR}/batched) only required for KokkosKernelsTargets.cmake in Trilinos?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That may be true but I am still at a loss for where these include directories are currently required.

If these lines can be deleted, even better. But to validate that, we should make this same change in the Trilinos copy of KokkosKernels and ensure that all of the KokkosKernels tests pass under Trilinos as well.

@ndellingwood
Copy link
Contributor

@e10harvey @bartlettroscoe thanks for raising these questions and the discussion, adding @lucbv so he is in the loop

The desired next steps, either here or a potential follow up PR, are unclear to me. For now, does the following sequence sound acceptable?

  1. We merge this PR as-is, as it matches the current patch to Trilinos that resolved the install testing issues
  2. A follow-on PR based on Evan's observations and suggestions

If I'm interpreting the discussion correctly, it sounds like there are a couple follow ups to consider:

  • Testing of Trilinos with changes to KokkosKernels that remove the KOKKOSKERNELS_INCLUDE_DIRECTORIES(REQUIRED_DURING_INSTALLATION_TESTING ${CMAKE_CURRENT_SOURCE_DIR}/* lines from the batched and blas directories; testing should confirm unit tests build and pass as expected, as well as install testing; if this works, we should consider getting these changes into kokkos-kernels for the 4.1 release
  • Longer term, build system refactorings based on Ross's comment

If I'm misunderstanding parts of the discussion, please correct me

@bartlettroscoe
Copy link
Contributor Author

For now, does the following sequence sound acceptable?

  1. We merge this PR as-is, as it matches the current patch to Trilinos that resolved the install testing issues
  2. A follow-on PR based on Evan's observations and suggestions

👍

@ndellingwood ndellingwood merged commit 6fdd658 into kokkos:develop Jun 14, 2023
ndellingwood added a commit that referenced this pull request Jun 14, 2023
…ncl-dir

KokkosKernels: Don't list include for non-existant 'batched' build dir (trilinos/Trilinos#11966)
(cherry picked from commit 6fdd658)
@ndellingwood
Copy link
Contributor

Cherry-picked to release-candidate-4.1.00, sha 7871bd2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CompatibilityUpdate TrilinosPatchMatch Apply this label for PR's mirroring changes submitted directly to Trilinos
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants