Code

Add GPU (CUDA) support#15

Code

Merged

jjleng merged 7 commits intomainjjleng/paka:mainfrom gpu-operatorjjleng/paka:gpu-operatorCopy head branch name to clipboardApr 10, 2024

+319-12

Commits

Commits on Apr 5, 2024

• [wip] gpu support

  jjleng

  committed
  95aa895

  Copy full SHA for 95aa895

Commits on Apr 10, 2024

• feat(gpu): run models on cuda GPUs

  jjleng

  committed
  b60a057

  Copy full SHA for b60a057

• feat(gpu): make nvidia device plugin tolerate model group taints

  jjleng

  committed
  195ed1e

  Copy full SHA for 195ed1e

• feat(gpu): set n_gpu_layers to offload work to gpu for the llama.cpp runtime

  jjleng

  committed
  81c034e

  Copy full SHA for 81c034e

• feat(gpu): larger disk for gpu nodes

  jjleng

  committed
  b1b39f7

  Copy full SHA for b1b39f7

• feat(gpu): make model group node disk size configerable

  jjleng

  committed
  864397c

  Copy full SHA for 864397c

• feat(gpu): be able to request a number of GPUs through config

  jjleng

  committed
  6a255ba

  Copy full SHA for 6a255ba