cusan-tests ·

Test environment for CUDA-aware MPI race detection.

Overview

• testsuite: Unit tests with data races
• jacobi Solver mini-app: CUDA-aware MPI C implementation by NVidia
• tealeaf mini-app: CUDA-aware MPI C++ heat conduction, see TeaLeaf

Requirements

CMakeLists facilitates the execution of our tests by generating appropriate targets.

To setup, execute:

$ cd cusan-tests
$ mkdir build && cd build
$ cmake ..

CMakeLists will require the following environment variables:

• CUSAN_PATH: Path to the cusan installation, to detect the compiler wrappers/libs
• MUST_PATH: PATH to the MUST installation, to detect mustrun

Software & GPU

• System modules: 1) gcc/11.2.0 2) cuda/11.8 3) openmpi/4.1.6 4) git/2.40.0 5) python/3.10.10 6) clang/14.0.6
• CuSan: bootstrap
• MUST: bootstrap
• testsuite: llvm-lit, FileCheck
• GPU: Tesla T4 and Tesla V100 (arch=sm_70)

testsuite Targets

In root level created build folder cmake .. generates:

• Targets to build, and run on the current node, see CMakeLists:
  • make check-cutests: All
  • make check-cutests-mpi-to-cuda: Only MPI to CUDA races
  • make check-cutests-cuda-to-mpi: Only CUDA to MPI races
  • make check-cutests-cuda-only: Only CUDA races

Jacobi Targets

In root level created build folder cmake .. generates:

• Targets to build, and run on the current node, see CMakeLists:
  • make jacobi-all-build: builds vanilla, vanilla-tsan, cusan
  • make jacobi-run or make jacobi-vanilla-run: Run on current node
• sbatch.sh: To run on compute node (requires make jacobi-all-build to be run)
  • make jacobi-sbatch: Slurm commit generated as sbatch

TeaLeaf Targets

In root level created build folder cmake .. generates:

• Targets to build, and run on the current node, see CMakeLists:
  • make tealeaf-all-build: builds vanilla, vanilla-tsan, cusan
  • make tealeaf-run or make tealeaf-vanilla-run: Run on current node
• sbatch.sh: To run on compute node (requires make tealeaf-all-build to be run)
  • make tealeaf-sbatch: Slurm commit generated as sbatch