Add TPL support for CBLAS and LAPACKE #554

jjwilke · 2019-12-20T00:32:26Z

No description provided.

mhoemmen · 2019-12-20T18:54:18Z

I've expressed my concerns about this in comments to PR #552. Summary: We're adding new TPLs for the sake of three functions that already exist in the BLAS and LAPACK TPLs. Downstream users will not enable these new TPLs unless they get automatically enabled.

ndellingwood · 2020-03-04T17:41:27Z

@jjwilke this build error occurred with cuda/10.1 on kokkos-dev-2 (volta70 arch, Cuda_OpenMP build) during spot-check:

/home/ndellin/kokkos-kernels/testing/cmTestAllSandia/TestAll_2020-03-03_16.09.02/cuda/10.1/Cuda_OpenMP-release/kokkos-install/include/Cuda/Kokkos_Cuda_Parallel.hpp(609): error: calling a __host__ function(" const") from a __device__ function("Kokkos::Impl::ParallelFor< ::,  ::Kokkos::RangePolicy< ::Kokkos::Cuda > ,  ::Kokkos::Cuda> ::operator () const") is not allowed

/home/ndellin/kokkos-kernels/testing/cmTestAllSandia/TestAll_2020-03-03_16.09.02/cuda/10.1/Cuda_OpenMP-release/kokkos-install/include/Cuda/Kokkos_Cuda_Parallel.hpp(609): error: identifier " const" is undefined in device code

/home/ndellin/kokkos-kernels/testing/cmTestAllSandia/TestAll_2020-03-03_16.09.02/cuda/10.1/Cuda_OpenMP-release/kokkos-install/include/Cuda/Kokkos_Cuda_Parallel.hpp(609): error: calling a __host__ function(" const") from a __device__ function("Kokkos::Impl::ParallelFor< ::,  ::Kokkos::RangePolicy< ::Kokkos::Cuda > ,  ::Kokkos::Cuda> ::operator () const") is not allowed

/home/ndellin/kokkos-kernels/testing/cmTestAllSandia/TestAll_2020-03-03_16.09.02/cuda/10.1/Cuda_OpenMP-release/kokkos-install/include/Cuda/Kokkos_Cuda_Parallel.hpp(609): error: identifier " const" is undefined in device code

4 errors detected in the compilation of "/tmp/tmpxft_0006f40f_00000000-6_KokkosSparse_spmv_struct_tuning.cpp1.ii".
make[2]: *** [perf_test/sparse/CMakeFiles/sparse_spmv_struct_tuning.dir/KokkosSparse_spmv_struct_tuning.cpp.o] Error 1
make[1]: *** [perf_test/sparse/CMakeFiles/sparse_spmv_struct_tuning.dir/all] Error 2

I retested with the kokkos-kernels develop branch to confirm these errors do not occur there as an extra sanity check.

jjwilke · 2020-03-04T19:40:11Z

I'm not able to reproduce this:

On kokkos-dev-2

./cm_test_all_sandia --kokkos-path=$HOME/Programs/kokkos/develop --spot-check cuda/10.1 --num=16

This builds and passes for me.

jjwilke · 2020-03-05T14:32:20Z

Okay, got it. spot-check-tpls not spot-check. I'm getting the error in both develop and this PR. The issue is that the CUSPARSE tests use lambdas, but CUDA_LAMBDA is not enabled for the TPL spot check. If I change cm_test_all_sandia to enable cuda lambda for the TPL test, everything passes.

I will open a separate PR that fixes this issue (and adds assertions) then come back to this.

jjwilke · 2020-03-05T14:55:31Z

Spot check failures addressed in PR #640

ndellingwood · 2020-03-05T15:32:49Z

@jjwilke

spot-check-tpls not spot-check

Looked back at my message, sorry I didn't say that.

I'm getting the error in both develop and this PR. The issue is that the CUSPARSE tests use lambdas

I didn't run into errors in develop when I was running the initial tpl spot-checks before submitting the PR to update the test script, which tests did these occur with? We should probably guard them like in kokkos.
I like the changes in your PR, thanks!

jjwilke · 2020-03-05T15:35:25Z

The build fails in KokkosSparse_spmv_struct_tuning.cpp. I actually got a Jenkins nightly e-mail with that error: Build failed in Jenkins: KokkosKernels_KokkosDev2_SPOTCHECK_TPLS #1

ndellingwood · 2020-03-05T15:36:17Z

I actually got a Jenkins nightly e-mail with that error

Got it, hadn't gotten to checking the Jenkins emails yet.

ndellingwood · 2020-03-05T15:56:28Z

@jjwilke PR #641 merged with fix to guard the failing spmv_struct perf test based on whether lambda support is enabled. Rerunning spot-check-tpls on this PR now.

ndellingwood · 2020-03-05T17:27:44Z

tpl spot-check with merge of #641 and all looks good:

 nohup ../../scripts/cm_test_all_sandia --spot-check-tpls
...

Running on machine: kokkos-dev-2

Repository Status:  d2cc0b7c91bb1905622bb3835be9fb65fbadb1fa Merge branch 'develop' into pr-554


Going to test compilers:  gcc/7.3.0 gcc/8.3.0 gcc/9.1 intel/18.0.5 clang/8.0 cuda/10.1
Testing compiler gcc/7.3.0
  Starting job gcc-7.3.0-OpenMP-release
kokkos options: 
kokkos devices: OpenMP
kokkos cxx: -O3 -Wall -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wignored-qualifiers -Wempty-body -Wclobbered -Wuninitialized 
  PASSED gcc-7.3.0-OpenMP-release
Testing compiler gcc/8.3.0
  Starting job gcc-7.3.0-Pthread-release
kokkos options: 
kokkos devices: Pthread
kokkos cxx: -O3 -Wall -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wignored-qualifiers -Wempty-body -Wclobbered -Wuninitialized 
  PASSED gcc-7.3.0-Pthread-release
Testing compiler gcc/9.1
  Starting job gcc-8.3.0-Serial-release
kokkos options: 
kokkos devices: Serial
kokkos cxx: -O3 -Wall -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wignored-qualifiers -Wempty-body -Wclobbered -Wuninitialized 
  PASSED gcc-8.3.0-Serial-release
  Starting job gcc-9.1-OpenMP-release
kokkos options: 
kokkos devices: OpenMP
kokkos cxx: -O3 -Wall -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wignored-qualifiers -Wempty-body -Wclobbered -Wuninitialized 
  PASSED gcc-9.1-OpenMP-release
Testing compiler intel/18.0.5
  Starting job gcc-9.1-Serial-release
kokkos options: 
kokkos devices: Serial
kokkos cxx: -O3 -Wall -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wignored-qualifiers -Wempty-body -Wclobbered -Wuninitialized 
  PASSED gcc-9.1-Serial-release
Testing compiler clang/8.0
  Starting job intel-18.0.5-OpenMP-release
kokkos options: 
kokkos devices: OpenMP
kokkos cxx: -O3 -Wall -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wuninitialized 
  PASSED intel-18.0.5-OpenMP-release
  Starting job clang-8.0-Cuda_OpenMP-release
kokkos options: 
kokkos devices: Cuda,OpenMP
kokkos cxx: -O3 -Wall -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wuninitialized -Wno-pass-failed 
  PASSED clang-8.0-Cuda_OpenMP-release
Testing compiler cuda/10.1
  Starting job clang-8.0-Pthread_Serial-release
kokkos options: 
kokkos devices: Pthread,Serial
kokkos cxx: -O3 -Wall -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wuninitialized -Wno-pass-failed 
  PASSED clang-8.0-Pthread_Serial-release
  Starting job cuda-10.1-Cuda_OpenMP-release
kokkos options: 
kokkos devices: Cuda,OpenMP
kokkos cxx: -O3 -Wall -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wuninitialized 
  PASSED cuda-10.1-Cuda_OpenMP-release
#######################################################
PASSED TESTS
#######################################################
clang-8.0-Cuda_OpenMP-release build_time=120 run_time=154
clang-8.0-Pthread_Serial-release build_time=52 run_time=154
cuda-10.1-Cuda_OpenMP-release build_time=160 run_time=135
gcc-7.3.0-OpenMP-release build_time=46 run_time=1477
gcc-7.3.0-Pthread-release build_time=44 run_time=96
gcc-8.3.0-Serial-release build_time=56 run_time=79
gcc-9.1-OpenMP-release build_time=54 run_time=63
gcc-9.1-Serial-release build_time=46 run_time=75
intel-18.0.5-OpenMP-release build_time=160 run_time=34

ndellingwood

@jjwilke sorry for delay, thought I had already approved. I'll go ahead and merge if that's okay?

jjwilke added the WIP Work In Progress label Dec 20, 2019

jjwilke changed the base branch from master to develop December 20, 2019 00:32

mhoemmen mentioned this pull request Dec 24, 2019

TSQR: Automatically detect CUBLAS & CUSOLVER TPLs; improve TPL handle wrappers trilinos/Trilinos#6496

Closed

mhoemmen mentioned this pull request Jan 14, 2020

TSQR: Automatically detect CUBLAS & CUSOLVER TPLs; improve TPL handle wrappers trilinos/Trilinos#6583

Merged

jjwilke force-pushed the tpl-cblas-lapacke branch from a656bff to c4cb5be Compare March 3, 2020 19:14

jjwilke mentioned this pull request Mar 3, 2020

Initial support for SuperLU TPL: Issue #545 #546

Closed

make sure cuda lambda support is enabled for cusparse tests

c97616e

TPL implementation of CBLAS/LAPACKE/SUPERLU/CHOLMOD

0fce0d3

jjwilke force-pushed the tpl-cblas-lapacke branch from 06974c2 to 0fce0d3 Compare March 5, 2020 15:48

ndellingwood mentioned this pull request Mar 5, 2020

KokkosSparse_spmv_struct_tuning: add lambda guard #641

Merged

ndellingwood approved these changes Mar 9, 2020

View reviewed changes

ndellingwood merged commit 8de01b1 into kokkos:develop Mar 10, 2020

kokkos-devops-admin mentioned this pull request Nov 2, 2021

Experimental tensor-core SpMV for BsrMatrix #1090

Merged

kokkos-devops-admin mentioned this pull request May 20, 2024

Interface for LAPACK geqrf() #2205

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add TPL support for CBLAS and LAPACKE #554

Add TPL support for CBLAS and LAPACKE #554

jjwilke commented Dec 20, 2019

mhoemmen commented Dec 20, 2019

ndellingwood commented Mar 4, 2020

jjwilke commented Mar 4, 2020

jjwilke commented Mar 5, 2020 •

edited

Loading

jjwilke commented Mar 5, 2020

ndellingwood commented Mar 5, 2020

jjwilke commented Mar 5, 2020

ndellingwood commented Mar 5, 2020

ndellingwood commented Mar 5, 2020

ndellingwood commented Mar 5, 2020

ndellingwood left a comment

Add TPL support for CBLAS and LAPACKE #554

Add TPL support for CBLAS and LAPACKE #554

Conversation

jjwilke commented Dec 20, 2019

mhoemmen commented Dec 20, 2019

ndellingwood commented Mar 4, 2020

jjwilke commented Mar 4, 2020

jjwilke commented Mar 5, 2020 • edited Loading

jjwilke commented Mar 5, 2020

ndellingwood commented Mar 5, 2020

jjwilke commented Mar 5, 2020

ndellingwood commented Mar 5, 2020

ndellingwood commented Mar 5, 2020

ndellingwood commented Mar 5, 2020

ndellingwood left a comment

Choose a reason for hiding this comment

jjwilke commented Mar 5, 2020 •

edited

Loading