Code

[Inference]Add CUDA KVCache Kernel#5406

Code

Merged

isky-cd merged 9 commits intohpcaitech:feature/colossal-inferhpcaitech/ColossalAI:feature/colossal-inferfrom isky-cd:cuda_kernel_branchisky-cd/ColossalAI:cuda_kernel_branchCopy head branch name to clipboardFeb 28, 2024

+348-75

Commits

Commits on Feb 27, 2024

• add cuda KVCache kernel

  isky-cd

  committed
  43ee989

  Copy full SHA for 43ee989

• annotation benchmark_kvcache_copy

  isky-cd

  committed
  55f611d

  Copy full SHA for 55f611d

Commits on Feb 28, 2024

• add use cuda

  isky-cd

  committed
  b4a2157

  Copy full SHA for b4a2157

• fix import path

  isky-cd

  committed
  6607316

  Copy full SHA for 6607316

• move benchmark scripts to example/

  isky-cd

  committed
  30b53aa

  Copy full SHA for 30b53aa

• rm benchmark codes in test_kv_cache_memcpy.py

  isky-cd

  committed
  4198898

  Copy full SHA for 4198898

• rm redundancy codes

  isky-cd

  committed
  f58d182

  Copy full SHA for f58d182

• rm redundancy codes

  isky-cd

  committed
  5c0178d

  Copy full SHA for 5c0178d

• pr was modified according to the review

  isky-cd

  committed
  4efb622

  Copy full SHA for 4efb622