cuda.sparse_spmv_tensor_core failures with clang/7+cuda/9

[ndellingwood]

@cwpearson when testing on kokkos-dev-2 after merge of #1307 I encountered test failures with spmv tensor code tests, can you investigate? I'm not sure if the failures are related to #1307 or showed up with separate changes

Output failure snip:

4: [ RUN      ] cuda.sparse_spmv_tensor_core_double_double_double_int_size_t_LayoutLeft_TestExecSpace
4: KokkosSparse::Test::spmv_tc: 16 errors of 16 for mv 0 (alpha=1, beta=0, mode = N)
4: /ascldap/users/ndellin/kokkos-kernels/unit_test/sparse/Test_Sparse_spmv.hpp:1306: Failure
4: Value of: num_errors == 0
4:   Actual: false
4: Expected: true
4: KokkosSparse::Test::spmv_tc: 16 errors of 16 for mv 1 (alpha=1, beta=0, mode = N)
4: /ascldap/users/ndellin/kokkos-kernels/unit_test/sparse/Test_Sparse_spmv.hpp:1306: Failure
4: Value of: num_errors == 0
4:   Actual: false
4: Expected: true
...

Reproducer (kokkos-dev-2):

module load sems-archive-env sems-archive-cmake/3.17.1 sems-archive-gcc/7.2.0 sems-archive-cuda/9.2 sems-archive-clang/7.0.1

$KOKKOSKERNELS_PATH/cm_generate_makefile.bash --with-devices=Cuda,OpenMP --arch=Volta70 --compiler=clang++ --cxxflags="-O3 -Wall -Wunused-parameter -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wuninitialized " --cxxstandard="14" --kokkos-path=$KOKKOS_PATH --kokkoskernels-path=$KOKKOSKERNELS_PATH --with-scalars='double,complex_double' --with-ordinals=int --with-offsets=int,size_t --with-layouts=LayoutLeft

[cwpearson]

First run

source /projects/sems/modulefiles/utils/sems-archive-modules-init.sh

[cwpearson]

I think this happens because here:

kokkos-kernels/src/Kokkos_ArithTraits.hpp

Line 655 in 2199f99

#if defined(KOKKOS_HALF_T_IS_FLOAT) && !KOKKOS_HALF_T_IS_FLOAT

Kokkos::ArithTraits for half is only properly defined when KOKKOS_HALF_T_IS_FLOAT = 0, otherwise, it uses float. This causes the half-precision tolerances to be too tight

[cwpearson]

Yes, prec() for half is actually float in CUDA 9 on the host.

[cwpearson]

And this may be the cause:

https://github.com/kokkos/kokkos/blob/542073765dcdb95a630fefe2547e5e309c7f3f3c/core/src/Cuda/Kokkos_Cuda_Half_Impl_Type.hpp#L50-L53

[cwpearson]

Hopefully fixed in #1322

[cwpearson]

A more comprehensive fix for the root issue is part of #1329