Windows install Guide in here

[Rakly3]

Install privateGPT Windows 10/11

Clone the repo

• git clone https://github.com/imartinez/privateGPT
• cd privateGPT

Create Conda env with Python 3.11

• conda create -n privateGPT python=3.11
• conda activate privateGPT

---

Install dependencies - in case you would still be missing dependencies, the error message in the terminal should tell you which one it is.

This can be entered as one command line

• pip install python-dateutil>=2.8.2 tornado>=6.1 decorator pygments>=2.4.0 huggingface_hub pydantic uvicorn llama_index fastapi injector chromadb python-multipart gradio llama-cpp-python transformers torch

---

Run in Windows Powershell terminal

• If Python installed from MS Store

  • (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python -

• If Python install not from MS Store

  • (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | py -

---

move file 'setup' to parent folder

• copy /privateGPT/scripts/setup => main folder /privateGPT/

---

some small tweaking

• go to private_gpt/ui/ and open file ui.py

• In the code look for upload_button = gr.components.UploadButton

  • Change the value
    • type="file" => type="filepath"

• in the terminal enter poetry run python -m private_gpt

  • to use other base than openAI paid API chatGPT
  • in the main folder /privateGPT
  • manually change the values in settings.yaml e.g. from 'mock' to 'local'
    • Or rename the files to make your choice settings.yaml file-name.

[kalle07]

thy ... but an error

why they can not make an easy install like text-generation-webui ;)

ok installl running all fine

error in powershell:
PS E:\privateGPT> C:\Users\kallemst\AppData\Roaming\Python\Scripts\poetry run python -m private_gpt
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "E:\privateGPT\private_gpt_main.py", line 3, in
import uvicorn
ModuleNotFoundError: No module named 'uvicorn'

error in ananconda setup (activated oc):
(privateGPT) E:\privateGPT>pip install python-dateutil>=2.8.2 tornado>=6.1 decorator pygments>=2.4.0 huggingface_hub pydantic uvicorn llama_index fastapi injector chromadb python-multipart gradio llama-cpp-python transformers torch

(privateGPT) E:\privateGPT>C:\Users\kallemst\AppData\Roaming\Python\Scripts\poetry run python -m private_gpt
21:15:45.661 [INFO ] private_gpt.settings.settings_loader - Starting application with profiles=['default']
21:15:49.650 [INFO ] private_gpt.ui.ui - Mounting the gradio UI, at path=/
21:15:49.730 [INFO ] uvicorn.error - Started server process [16232]
21:15:49.730 [INFO ] uvicorn.error - Waiting for application startup.
21:15:49.730 [INFO ] uvicorn.error - Application startup complete.
21:15:49.732 [INFO ] uvicorn.error - Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)

may 0.0.0.0 dont work on my machine, where i can change it to 127.0.0.1:8888
is some what else going wrong ?

OK SRY
if i manual 127.0.0.1:8001
the the browwser adress work but i cant do anything ...
no model (i hve copy one gguf inside "model" folder ...

must i install some api stuff
pip install gradio_client ???

THX

[doobidoo]

The following section seems to be obsolete now:

go to private_gpt/ui/ and open file ui.py

In the code look for upload_button = gr.components.UploadButton

Change the value
    type="file" => type="filepath"

[TomLucidor]

Why so? And what other tweaks should be anticipated?

[TomLucidor]

Notes on using pyenv-win:

:: [Info] ::  Mirror: https://www.python.org/ftp/python
pyenv-install: definition not found: 3.11

See all available versions with `pyenv install --list'.

[TomLucidor]

Remember to install the needed software through Chocolatey (so now it is 5-step process)

choco install git anaconda3

(Need to add PATH here)

git clone https://github.com/imartinez/privateGPT
cd privateGPT
conda create -n privateGPT python=3.11
conda activate privateGPT
pip install python-dateutil>=2.8.2 tornado>=6.1 decorator pygments>=2.4.0 huggingface_hub pydantic uvicorn llama_index fastapi injector chromadb python-multipart gradio llama-cpp-python transformers torch

(Install Poetry through pip, YAML file currently defaults to "local" btw so no need to sweat)

Side note: Oobabooga solved the llama-cpp-python issue with oobabooga/text-generation-webui#1534 (comment)
See also: #336 #974
Addendum: WAIT is this is a versioning issue? #683

Note 1: use Poetry that is installed through pip instead of (Invoke-WebRequest -Uri https://install.python-poetry.org/ -UseBasicParsing).Content | py - due to some kind of instability
Note 2: Only do poetry run python -m private_gpt after poetry install --with ui and poetry run python scripts/setup https://docs.privategpt.dev/installation

[kaas1551]

Hi.
I seem to run into an error regarding llama-cpp-python and CMake.
im getting the following on Win 10:

error: subprocess-exited-with-error
× Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [20 lines of output]
*** scikit-build-core 0.8.2 using CMake 3.28.3 (wheel)
*** Configuring CMake...
2024-03-18 05:18:51,019 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None
loading initial cache file C:\Users\merij\AppData\Local\Temp\tmpo9sym148\build\CMakeInit.txt
-- Building for: NMake Makefiles
CMake Error at CMakeLists.txt:3 (project):
Running
'nmake' '-?'
failed with:
The system cannot find the file specified
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
*** CMake configuration failed
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

[kalle07]

again why such ascomplicate (Chocolatey?)
i mean you can copy and paste from other that works much simpler and faster

spy on Jan he has made it ^^

[kaas1551]

What do you mean?

[kalle07]

sry, I meant that there is so much open source code here on Git, why don't you copy and paste or get an idea of how they did it ^^
like:
kobold
jan
lm-ai
oobadooga
...

[KarthikGowdaRamakrishna]

Same here i think

Building wheels for collected packages: llama-cpp-python, ffmpy
Building wheel for llama-cpp-python (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [20 lines of output]
*** scikit-build-core 0.8.2 using CMake 3.28.3 (wheel)
*** Configuring CMake...
2024-03-19 15:16:05,179 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None
loading initial cache file C:\Users\pavan\AppData\Local\Temp\tmp1hehjnor\build\CMakeInit.txt
-- Building for: NMake Makefiles
CMake Error at CMakeLists.txt:3 (project):
Running

     'nmake' '-?'

    failed with:

     The system cannot find the file specified


  CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
  CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
  -- Configuring incomplete, errors occurred!

  *** CMake configuration failed
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for llama-cpp-python
Building wheel for ffmpy (setup.py) ... done
Created wheel for ffmpy: filename=ffmpy-0.3.2-py3-none-any.whl size=5605 sha256=62b33a049b9507c5027415a5819e2726da75f8176e170261750cbf62aaba905a
Stored in directory: c:\users\pavan\appdata\local\pip\cache\wheels\55\3c\f2\f6e34046bac0d57c13c7d08123b85872423b89c8f59bafda51
Successfully built ffmpy
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

[kaas1551]

Make sure you install visual studio with the C++ workload aswell as CMake. This ended up fixing this particular issue for me. I still ended up running into other unrelated issues regardless of this, so I'm running my LLM's directly through Ollama (since i only needed the API and dont care for web ui)

[KarthikGowdaRamakrishna]

I'm pretty new to all this , sorry to ask this but how to install it? i tried to install CMake plugin on VS code extensions

[kaas1551]

VSCode is a code editor, entirely separate from Visual Studio, the IDE. You need to install visual studio. You also need MinGW. Please have a look here:

https://docs.privategpt.dev/installation/getting-started/installation#llama-cpp-known-issues-and-troubleshooting

[JTMarsh556]

This works for me: Follow this video up until 6:07 https://www.youtube.com/watch?v=hfnxVlPGLZs. Then pick up here for ollama (I cant get good performance on this right now): https://docs.privategpt.dev/installation/getting-started/installation#local-ollama-powered-setup-recommended or here for for llama (works way better for me right now) https://docs.privategpt.dev/installation/getting-started/installation#llama-cpp-support. For CUDA support refer to the video first mentioned at 9:58 or https://docs.privategpt.dev/installation/getting-started/installation#llama-cpp-windows-nvidia-gpu-support. This will work if you follow all of the steps.