Test failures

[alexandermorozov]

CUDA tests for dot and nrm2 fail. Rarely other CUDA-related tests fail too.

• OS: Debian Testing, Linux 4.5
• Driver: NVIDIA UNIX x86_64 Kernel Module 352.79 Wed Jan 13 16:17:53 PST 2016
• branch: master, up-to-date

dmesg:

[  170.053881] NVRM: Xid (PCI:0000:01:00): 31, Ch 00000013, engmask 00000101, intr 10000000
[  170.074502] NVRM: Xid (PCI:0000:01:00): 31, Ch 00000012, engmask 00000101, intr 10000000
[  391.163575] NVRM: Xid (PCI:0000:01:00): 31, Ch 00000019, engmask 00000101, intr 10000000
[  397.859409] NVRM: Xid (PCI:0000:01:00): 31, Ch 00000018, engmask 00000101, intr 10000000
[  402.360207] NVRM: Xid (PCI:0000:01:00): 31, Ch 00000019, engmask 00000101, intr 10000000
[  409.619714] NVRM: Xid (PCI:0000:01:00): 31, Ch 0000001b, engmask 00000101, intr 10000000
[  469.945853] NVRM: Xid (PCI:0000:01:00): 31, Ch 0000001b, engmask 00000101, intr 10000000
[  469.956065] NVRM: Xid (PCI:0000:01:00): 31, Ch 00000019, engmask 00000101, intr 10000000

Xid 31 == GPU memory page fault

butler@antiferno:~/packs/collenchyma-blas$ cargo test   
     Running target/debug/blas_specs-1f948e68660c7d92

running 26 tests
test blas_spec::cuda::it_computes_correct_asum_on_cuda_for_f32 ... ok
test blas_spec::cuda::it_computes_correct_axpy_on_cuda_for_f32 ... ok
test blas_spec::cuda::it_computes_correct_copy_on_cuda_for_f32 ... ok
test blas_spec::cuda::it_computes_correct_dot_on_cuda_for_f32 ... FAILED
test blas_spec::cuda::it_computes_correct_gemm_on_cuda_for_f32 ... FAILED
test blas_spec::cuda::it_computes_correct_nrm2_on_cuda_for_f32 ... FAILED
test blas_spec::native::it_computes_correct_asum_on_native_for_f32 ... ok
test blas_spec::native::it_computes_correct_asum_on_native_for_f64 ... ok
test blas_spec::native::it_computes_correct_axpy_on_native_for_f32 ... ok
test blas_spec::native::it_computes_correct_axpy_on_native_for_f64 ... ok
test blas_spec::native::it_computes_correct_copy_on_native_for_f32 ... ok
test blas_spec::native::it_computes_correct_copy_on_native_for_f64 ... ok
test blas_spec::native::it_computes_correct_dot_on_native_for_f32 ... ok
test blas_spec::native::it_computes_correct_dot_on_native_for_f64 ... ok
test blas_spec::native::it_computes_correct_gemm_on_native_for_f32 ... ok
test blas_spec::native::it_computes_correct_gemm_on_native_for_f64 ... ok
test blas_spec::native::it_computes_correct_gemm_transpose_on_native_for_f32 ... ok
test blas_spec::native::it_computes_correct_nrm2_on_native_for_f32 ... ok
test blas_spec::native::it_computes_correct_nrm2_on_native_for_f64 ... ok
test blas_spec::native::it_computes_correct_scal_on_native_for_f32 ... ok
test blas_spec::native::it_computes_correct_scal_on_native_for_f64 ... ok
test blas_spec::native::it_computes_correct_swap_on_native_for_f32 ... ok
test blas_spec::native::it_computes_correct_swap_on_native_for_f64 ... ok
test blas_spec::cuda::it_computes_correct_scal_on_cuda_for_f32 ... ok
test blas_spec::cuda::it_computes_correct_swap_on_cuda_for_f32 ... ok
test blas_spec::cuda::it_computes_correct_transpose_gemm_on_cuda_for_f32 ... ok

failures:

---- blas_spec::cuda::it_computes_correct_dot_on_cuda_for_f32 stdout ----
    thread 'blas_spec::cuda::it_computes_correct_dot_on_cuda_for_f32' panicked at 'called `Result::unwrap()` on an `Err` value: Plugin(Operation("Unable to execute operation dot."))', ../src/libcore/result.rs:785
stack backtrace:
   1:     0x55e7c5289a20 - std::sys::backtrace::tracing::imp::write::h9fb600083204ae7f
   2:     0x55e7c528ce7b - std::panicking::default_hook::_$u7b$$u7b$closure$u7d$$u7d$::hca543c34f11229ac
   3:     0x55e7c528ca6c - std::panicking::default_hook::hc2c969e7453d080c
   4:     0x55e7c528050f - std::sys_common::unwind::begin_unwind_inner::h30e12d15ce2b2e25
   5:     0x55e7c52814d8 - std::sys_common::unwind::begin_unwind_fmt::hb2de8a9968d38523
   6:     0x55e7c5288c71 - rust_begin_unwind
   7:     0x55e7c52c2f3f - core::panicking::panic_fmt::h257ceb0aa351d801
   8:     0x55e7c51dd348 - core::result::unwrap_failed::h0d8273a186b5b477
                        at ../src/libcore/macros.rs:29
   9:     0x55e7c51e4e06 - _<std..result..Result<T, E>>::unwrap::hd26718a3964659af
                        at ../src/libcore/result.rs:723
  10:     0x55e7c51fb6ed - blas_specs::blas_spec::cuda::it_computes_correct_dot_on_cuda_for_f32::h13d7ffbb2d118a9e
                        at tests/blas_specs.rs:428
  11:     0x55e7c521b096 - _<F as std..boxed..FnBox<A>>::call_box::hebdedc6b66e381cc
  12:     0x55e7c521d7db - std::sys_common::unwind::try::try_fn::h7c02806ebfbd3142
  13:     0x55e7c5288bfb - __rust_try
  14:     0x55e7c5288b8d - std::sys_common::unwind::inner_try::h47a4d9cd4a369dcd
  15:     0x55e7c521db5a - _<F as std..boxed..FnBox<A>>::call_box::h214ad3593915cb36
  16:     0x55e7c528b524 - std::sys::thread::Thread::new::thread_start::h6f266e069bf4ec2b
  17:     0x7fad17525453 - start_thread
  18:     0x7fad1704cecc - clone
  19:                0x0 - <unknown>

---- blas_spec::cuda::it_computes_correct_gemm_on_cuda_for_f32 stdout ----
    thread 'blas_spec::cuda::it_computes_correct_gemm_on_cuda_for_f32' panicked at 'assertion failed: `(left == right)` (left: `[28, 7, 7, 28, 7, 7, 28, 7, 7]`, right: `[392, 98, 98, 392, 98, 98, 392, 98, 98]`)', tests/blas_specs.rs:488
stack backtrace:
   1:     0x55e7c5289a20 - std::sys::backtrace::tracing::imp::write::h9fb600083204ae7f
   2:     0x55e7c528ce7b - std::panicking::default_hook::_$u7b$$u7b$closure$u7d$$u7d$::hca543c34f11229ac
   3:     0x55e7c528ca6c - std::panicking::default_hook::hc2c969e7453d080c
   4:     0x55e7c528050f - std::sys_common::unwind::begin_unwind_inner::h30e12d15ce2b2e25
   5:     0x55e7c52814d8 - std::sys_common::unwind::begin_unwind_fmt::hb2de8a9968d38523
   6:     0x55e7c51fd77f - blas_specs::blas_spec::cuda::it_computes_correct_gemm_on_cuda_for_f32::h819da5f7911be170
                        at /home/butler/packs/collenchyma-blas/<std macros>:8
   7:     0x55e7c521b096 - _<F as std..boxed..FnBox<A>>::call_box::hebdedc6b66e381cc
   8:     0x55e7c521d7db - std::sys_common::unwind::try::try_fn::h7c02806ebfbd3142
   9:     0x55e7c5288bfb - __rust_try
  10:     0x55e7c5288b8d - std::sys_common::unwind::inner_try::h47a4d9cd4a369dcd
  11:     0x55e7c521db5a - _<F as std..boxed..FnBox<A>>::call_box::h214ad3593915cb36
  12:     0x55e7c528b524 - std::sys::thread::Thread::new::thread_start::h6f266e069bf4ec2b
  13:     0x7fad17525453 - start_thread
  14:     0x7fad1704cecc - clone
  15:                0x0 - <unknown>

---- blas_spec::cuda::it_computes_correct_nrm2_on_cuda_for_f32 stdout ----
    thread 'blas_spec::cuda::it_computes_correct_nrm2_on_cuda_for_f32' panicked at 'called `Result::unwrap()` on an `Err` value: Plugin(Operation("Unable to execute operation nrm2."))', ../src/libcore/result.rs:785
stack backtrace:
   1:     0x55e7c5289a20 - std::sys::backtrace::tracing::imp::write::h9fb600083204ae7f
   2:     0x55e7c528ce7b - std::panicking::default_hook::_$u7b$$u7b$closure$u7d$$u7d$::hca543c34f11229ac
   3:     0x55e7c528ca6c - std::panicking::default_hook::hc2c969e7453d080c
   4:     0x55e7c528050f - std::sys_common::unwind::begin_unwind_inner::h30e12d15ce2b2e25
   5:     0x55e7c52814d8 - std::sys_common::unwind::begin_unwind_fmt::hb2de8a9968d38523
   6:     0x55e7c5288c71 - rust_begin_unwind
   7:     0x55e7c52c2f3f - core::panicking::panic_fmt::h257ceb0aa351d801
   8:     0x55e7c51dd348 - core::result::unwrap_failed::h0d8273a186b5b477
                        at ../src/libcore/macros.rs:29
   9:     0x55e7c51e4e06 - _<std..result..Result<T, E>>::unwrap::hd26718a3964659af
                        at ../src/libcore/result.rs:723
  10:     0x55e7c51fbcf0 - blas_specs::blas_spec::cuda::it_computes_correct_nrm2_on_cuda_for_f32::h58eee3465d0af9f2
                        at tests/blas_specs.rs:443
  11:     0x55e7c521b096 - _<F as std..boxed..FnBox<A>>::call_box::hebdedc6b66e381cc
  12:     0x55e7c521d7db - std::sys_common::unwind::try::try_fn::h7c02806ebfbd3142
  13:     0x55e7c5288bfb - __rust_try
  14:     0x55e7c5288b8d - std::sys_common::unwind::inner_try::h47a4d9cd4a369dcd
  15:     0x55e7c521db5a - _<F as std..boxed..FnBox<A>>::call_box::h214ad3593915cb36
  16:     0x55e7c528b524 - std::sys::thread::Thread::new::thread_start::h6f266e069bf4ec2b
  17:     0x7fad17525453 - start_thread
  18:     0x7fad1704cecc - clone
  19:                0x0 - <unknown>


failures:
    blas_spec::cuda::it_computes_correct_dot_on_cuda_for_f32
    blas_spec::cuda::it_computes_correct_gemm_on_cuda_for_f32
    blas_spec::cuda::it_computes_correct_nrm2_on_cuda_for_f32

test result: FAILED. 23 passed; 3 failed; 0 ignored; 0 measured

error: test failed