README.md

[EXPERIMENTAL] TPP GPU support

Experimental support for GPU code generation within TPP-MLIR.

Note: the GPU support is disabled by default. It is an opt-in feature which can be enabled with CMake flag -DTPP_GPU=cuda|vulkan.

LLVM with GPU support

Build LLVM with GPU support:

cd llvm-project/build

cmake -G Ninja ../llvm \
   -DLLVM_ENABLE_PROJECTS=mlir \
   -DLLVM_BUILD_EXAMPLES=ON \
   -DLLVM_INSTALL_UTILS=ON \
   -DLLVM_TARGETS_TO_BUILD="host;NVPTX;AMDGPU" \
   -DCMAKE_BUILD_TYPE=RelWithDebInfo \
   -DLLVM_ENABLE_ASSERTIONS=ON \
   -DCMAKE_C_COMPILER=clang \
   -DCMAKE_CXX_COMPILER=clang++ \
   -DCMAKE_CUDA_COMPILER=nvcc \
   -DMLIR_ENABLE_CUDA_RUNNER=ON \
   -DMLIR_ENABLE_CUDA_CONVERSIONS=ON \
   -DMLIR_ENABLE_SPIRV_CPU_RUNNER=ON \
   -DMLIR_ENABLE_VULKAN_RUNNER=ON

TPP MLIR with GPU support

This setup assumes that the LLVM with GPU support is set as $CUSTOM_LLVM_ROOT.

The GPU support is disabled by default and can be enabled with the CMake flag -DTPP_GPU=cuda|vulkan.

cd tpp-mlir/build

cmake -G Ninja .. \
   -DCMAKE_BUILD_TYPE=RelWithDebInfo \
   -DMLIR_DIR=$CUSTOM_LLVM_ROOT/lib/cmake/mlir \
   -DLLVM_EXTERNAL_LIT=$CUSTOM_LLVM_ROOT/bin/llvm-lit \
   -DCMAKE_C_COMPILER=clang \
   -DCMAKE_CXX_COMPILER=clang++ \
   -DTPP_GPU="cuda;vulkan"

CUDA setup

• Install CUDA on WSL 2
  1. HOST - install newest NVIDIA drivers on host
  2. WSL - remove the old GPG key: sudo apt-key del 7fa2af80
  3. WSL - follow instructions to install x86 CUDA Toolkit
• CUDA samples - useful to test CUDA installation

Troubleshooting

• CUDA driver vs runtime version mismatch:
  • check if CUDA driver and cuda-toolkit versions are the same:

    nvidia-smi
    dpkg -l | grep cuda-toolkit

  • thread with more details - link
• MLIR CUDA_ERROR_ILLEGAL_ADDRESS bug - link

Notes

• Monitor GPU usage

  watch -n 0.1 nvidia-smi