Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FindTPLROC*: updates to fix export of import targets #2250

Merged
merged 1 commit into from
Jun 18, 2024
Merged

FindTPLROC*: updates to fix export of import targets #2250

merged 1 commit into from
Jun 18, 2024

Conversation

ndellingwood
Copy link
Contributor

Changes for the Rocm tpls to match the handling as done with the Cuda tpls

Should resolve issue #2238

@ndellingwood
FindTPLROC*: updates to fix export of import targets
819c40b 
Changes for the Rocm tpls to match the handling as done with the Cuda tpls

Should resolve issue kokkos#2238
@ndellingwood
Copy link
Contributor Author

ndellingwood commented Jun 17, 2024 •
edited
Loading

I ran several tests with the changes:

  • Kokkos Kernels unit tests with rocm tpls enabled - in this posted configuration, I tested multiple paths within the same configuration (rocblas: providing the include, libary_dirs and librarynames; rocsparse: providing the lib root; rocsolver: no additional info specified)
# Setup environment
module purge
module load python rocm/5.6.1 cmake openmpi/4.1.5 openblas/0.3.23 ninja
module list
export OMPI_CXX=$ROCM_PATH/bin/hipcc

# Kokkos:
  cmake \
    -D CMAKE_CXX_COMPILER="$(which mpicxx)" \
    -D CMAKE_C_COMPILER="$(which mpicc)" \
    -D CMAKE_INSTALL_PREFIX=$KOKKOS_INSTALL \
    -D CMAKE_BUILD_TYPE:STRING=RELEASE \
    -D BUILD_SHARED_LIBS:BOOL=OFF \
  \
    -D Kokkos_ENABLE_SERIAL=ON \
    -D Kokkos_ENABLE_HIP=ON \
     -D Kokkos_ENABLE_TESTS=OFF \
    -D Kokkos_ARCH_VEGA90A=ON \
    -D Kokkos_ENABLE_COMPLEX_ALIGN=OFF \
    -D Kokkos_ENABLE_ROCTHRUST=OFF \
  \
  $KOKKOS_DIR

# Kernels:
  cmake \
    -D CMAKE_CXX_COMPILER="$(which mpicxx)" \
    -D CMAKE_C_COMPILER="$(which mpicc)" \
    -D CMAKE_INSTALL_PREFIX=$KOKKOSKERNELS_INSTALL \
    -D CMAKE_BUILD_TYPE:STRING=RELEASE \
    -D BUILD_SHARED_LIBS:BOOL=OFF \
  \
    -D Kokkos_ROOT=$KOKKOS_INSTALL \
  \
    -D KokkosKernels_ENABLE_TPL_BLAS=ON \
    -D KokkosKernels_ENABLE_TPL_LAPACK=ON \
    -D KokkosKernels_ENABLE_TPL_ROCBLAS=ON \
     -D ROCBLAS_LIBRARIES=-lrocblas \
     -D ROCBLAS_LIBRARY_DIRS=${ROCM_ROOT}/rocblas/lib \
     -D ROCBLAS_INCLUDE_DIRS=${ROCM_ROOT}/rocblas/include \
    -D KokkosKernels_ENABLE_TPL_ROCSPARSE=ON \
     -D ROCSPARSE_ROOT=${ROCM_ROOT}/rocsparse \
    -D KokkosKernels_ENABLE_TPL_ROCSOLVER=ON \
  \
    -D KokkosKernels_INST_COMPLEX_DOUBLE=ON \
    -D KokkosKernels_INST_OFFSET_INT=OFF \
    -D KokkosKernels_INST_OFFSET_SIZE_T=ON \
    -D KokkosKernels_ENABLE_TESTS=ON \
  \
  $KOKKOSKERNELS_DIR
  • unit tests passed 100%

KokkosKernelsTargets.cmake (snip):

...
set_target_properties(Kokkos::ROCBLAS PROPERTIES
  INTERFACE_INCLUDE_DIRECTORIES "/opt/rocm-5.6.1/rocblas/include"
  INTERFACE_LINK_LIBRARIES "/opt/rocm-5.6.1/rocblas/lib/librocblas.so"
)

# Create imported target Kokkos::ROCSPARSE
add_library(Kokkos::ROCSPARSE INTERFACE IMPORTED)

set_target_properties(Kokkos::ROCSPARSE PROPERTIES
  INTERFACE_INCLUDE_DIRECTORIES "/opt/rocm-5.6.1/rocsparse/include/"
  INTERFACE_LINK_LIBRARIES "/opt/rocm-5.6.1/rocsparse/lib/librocsparse.so"
)

# Create imported target Kokkos::ROCSOLVER
add_library(Kokkos::ROCSOLVER INTERFACE IMPORTED)

set_target_properties(Kokkos::ROCSOLVER PROPERTIES
  INTERFACE_INCLUDE_DIRECTORIES "/opt/rocm-5.6.1/rocsolver/include/"
  INTERFACE_LINK_LIBRARIES "/opt/rocm-5.6.1/rocsolver/lib/librocsolver.so"
)

# Create imported target Kokkos::kokkoskernels
add_library(Kokkos::kokkoskernels STATIC IMPORTED)

set_target_properties(Kokkos::kokkoskernels PROPERTIES
  INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include"
  INTERFACE_LINK_LIBRARIES "Kokkos::kokkos;Kokkos::LAPACK;Kokkos::BLAS;Kokkos::ROCBLAS;Kokkos::ROCSPARSE;Kokkos::ROCSOLVER"
)
...
  • Trilinos using kokkos and kokkos-kernels as external TPLs with rocm tpls enabled - successfully configured, compiled and Tpetra, Sacado, Stokhos tests pass 100%

@kokkos-devops-admin
Copy link

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request.

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight

  • Build Num: 1388
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10

  • Build Num: 976
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GNU1021

  • Build Num: 633
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GNU1021_Light_LayoutRight

  • Build Num: 620
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_GNU1021

  • Build Num: 621
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_INTEL19_solo

  • Build Num: 625
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_CLANG1001_solo

  • Build Num: 597
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_VEGA90A_ROCM561

  • Build Num: 1078
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_VEGA90A_Tpls_ROCM561

  • Build Num: 593
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Using Repos:

Repo: KOKKOSKERNELS (ndellingwood/kokkos-kernels)
  • Branch: rocmtpl-export-targets
  • SHA: 819c40b
  • Mode: TEST_REPO

Pull Request Author: ndellingwood

@brian-kelley
Copy link
Contributor

This LGTM. It looks like we could probably wrap this logic into a function and reuse it in all the rocm/cuda TPLs but it doesn't have to be in this PR.

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight

  • Build Num: 1388
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10

  • Build Num: 976
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GNU1021

  • Build Num: 633
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GNU1021_Light_LayoutRight

  • Build Num: 620
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_GNU1021

  • Build Num: 621
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_INTEL19_solo

  • Build Num: 625
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_CLANG1001_solo

  • Build Num: 597
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_VEGA90A_ROCM561

  • Build Num: 1078
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_VEGA90A_Tpls_ROCM561

  • Build Num: 593
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/ndellingwood/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 819c40b
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA fe2a92f
PR_LABELS
PULLREQUESTNUM 2250
TEST_REPO_ALIAS KOKKOSKERNELS

@kokkos-devops-admin
Copy link

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ brian-kelley ]!

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge

@ndellingwood ndellingwood merged commit b91e258 into kokkos:develop Jun 18, 2024
16 checks passed
@ndellingwood ndellingwood deleted the rocmtpl-export-targets branch June 18, 2024 03:01
@jgfouca
Copy link
Contributor

jgfouca commented Jun 18, 2024

Thanks, @ndellingwood !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brian-kelley brian-kelley brian-kelley approved these changes

@jgfouca jgfouca Awaiting requested review from jgfouca

@lucbv lucbv Awaiting requested review from lucbv

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ndellingwood @kokkos-devops-admin @brian-kelley @jgfouca