This repository contains a set of demo scripts to show how you can use the Notus 7B large language model. Please follow the instructions below to get started.
Hardware requirements:
- Nvidia GPU with 16GB memory (e.g. RTX4080)
- 32GB RAM
Software requirements:
- Python 3.11
- Linux (WSL2 works too)
Follow these steps to configure your machine:
git clone https://github.com/wmeints/notus-model-demo.git
cd notus-model-demo
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
Next, try one of the demos:
- Streaming:
python stream_response.py
- Non-streaming:
python generate_response.py