Skip to content

Commit

Permalink
koboldcpp.sh - The Mamba Multitool (#554)
Browse files Browse the repository at this point in the history
* .sh script V1

* koboldcpp.sh polish

* koboldcpp.sh dist generator

* Include html's in dist

* RWKV in Linux Dist

* Lower dependency requirements

* Eliminate wget dependency

* More distinct binary name

I know its technically amd64, but I don't want to cause confusion among nvidia users.

* Use System OpenCL

Unsure how this will behave in the pyinstaller build, but pocl ended up CPU only. With a bit of luck the pyinstaller uses the one from the actual system if compiled in a system without opencl, while conda now includes it for that specific system.

* Add cblas dependency

Missing this causes compile failures on some system's

* ICD workaround

Ideally we find a better solution, but conda forces ICD and needs this for the successful compile. However, pyinstaller then embeds the ICD causing it to be limited to the system it was compiled for. By temporarily removing the ICD pyinstaller can't find it and everything remains functional. Ideally we do this on a pyinstaller level, but I could not find any good options to do so yet.

---------

Co-authored-by: root <root@DESKTOP-DQ1QRAG>
  • Loading branch information
henk717 and root authored Dec 10, 2023
1 parent 7a69152 commit e2cf3b7
Show file tree
Hide file tree
Showing 3 changed files with 49 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -118,5 +118,8 @@ hipblas.dll
koboldcpp_hipblas.so
koboldcpp_hipblas.dll

bin/
conda/

# Jetbrains idea folder
.idea/
24 changes: 24 additions & 0 deletions environment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: koboldcpp
channels:
- nvidia/label/cuda-11.5.0
- conda-forge
- defaults
dependencies:
- python=3.8
- cuda-nvcc=11.5
- cuda-libraries-dev=11.5
- cxx-compiler
- gxx=10
- pip
- git=2.35.1
- libopenblas
- openblas
- clblast
- ninja
- make
- packaging
- pyinstaller
- libcblas
- ocl-icd-system
- pip:
- customtkinter
22 changes: 22 additions & 0 deletions koboldcpp.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/bin/bash
if [ ! -f "bin/micromamba" ]; then
curl -Ls https://anaconda.org/conda-forge/micromamba/1.5.3/download/linux-64/micromamba-1.5.3-0.tar.bz2 | tar -xvj bin/micromamba
fi

if [[ ! -f "conda/envs/linux/bin/python" || $1 == "rebuild" ]]; then
bin/micromamba create --no-shortcuts -r conda -n linux -f environment.yaml -y
bin/micromamba create --no-shortcuts -r conda -n linux -f environment.yaml -y
bin/micromamba run -r conda -n linux make clean
fi

bin/micromamba run -r conda -n linux make LLAMA_OPENBLAS=1 LLAMA_CLBLAST=1 LLAMA_CUBLAS=1 LLAMA_PORTABLE=1

if [[ $1 == "rebuild" ]]; then
echo Rebuild complete, you can now try to launch Koboldcpp.
elif [[ $1 == "dist" ]]; then
bin/micromamba remove -r conda -n linux --force ocl-icd -y
bin/micromamba run -r conda -n linux pyinstaller --noconfirm --onefile --collect-all customtkinter --add-data='./koboldcpp_default.so:.' --add-data='./koboldcpp_cublas.so:.' --add-data='./koboldcpp_openblas.so:.' --add-data='./koboldcpp_clblast.so:.' --add-data='./koboldcpp_clblast_noavx2.so:.' --add-data='./klite.embd:.' --add-data='./kcpp_docs.embd:.' --add-data='./rwkv_vocab.embd:.' --add-data='./rwkv_world_vocab.embd:.' --clean --console koboldcpp.py -n "koboldcpp-linux-x64"
bin/micromamba install -r conda -n linux ocl-icd -c conda-forge -y
else
bin/micromamba run -r conda -n linux python koboldcpp.py $*
fi

0 comments on commit e2cf3b7

Please sign in to comment.