User Preferences and Setup Docs

.env.sample

@@ -7,12 +7,18 @@ GUILD_ID = GUILD_ID
 # Channel where the bot listens to messages
 CHANNEL_ID = CHANNEL_ID
 
-# model for the bot to query from (i.e. llama2 [llama2:13b], mistral, ... )
+# model for the bot to query from (i.e. llama2 [llama2:13b], mistral, codellama, etc... )
 MODEL = MODEL_NAME
 
 # discord bot user id for mentions
 CLIENT_UID = BOT_USER_ID
 
-# ip address of docker container, this will have to be found manually (docker can also change it too)
+# ip/port address of docker container, I use 172.18.X.X for docker, 127.0.0.1 for local
 OLLAMA_IP = IP_ADDRESS
-OLLAMA_PORT = PORT
+OLLAMA_PORT = PORT
+
+# ip address for discord bot container, I use 172.18.X.X, use different IP than ollama_ip
+DISCORD_IP = IP_ADDRESS
+
+# subnet address, ex. 172.18.0.0 as we use /16.
+SUBNET_ADDRESS = ADDRESS

.gitignore

@@ -1,6 +1,9 @@
 # Created by https://www.toptal.com/developers/gitignore/api/node
 # Edit at https://www.toptal.com/developers/gitignore?templates=node
 
+# config
+config.json
+
 # builds
 build/
 dist/

Dockerfile

@@ -7,6 +7,7 @@ WORKDIR /app
 # copy package.json and the lock file into the container, and src files
 COPY ./src ./src
 COPY ./*.json ./
+COPY ./.env ./
 
 # install dependencies, breaks
 RUN npm install

README.md

@@ -1,39 +1,14 @@
 # Discord Ollama Integration [![License: CC BY-NC 4.0](https://img.shields.io/badge/License-CC_BY--NC_4.0-darkgreen.svg)](https://creativecommons.org/licenses/by-nc/4.0/) [![Release Badge](https://img.shields.io/github/v/release/kevinthedang/discord-ollama?logo=github)](https://github.com/kevinthedang/discord-ollama/releases/latest)
 Ollama is an AI model management tool that allows users to install and use custom large language models locally. The goal is to create a discord bot that will utilize Ollama and chat with it on a Discord!
 
-## Ollama Setup
-* Go to Ollama's [Linux download page](https://ollama.ai/download/linux) and run the simple curl command they provide. The command should be `curl https://ollama.ai/install.sh | sh`.
-* Now the the following commands in separate terminals to test out how it works!
-    * In terminal 1 -> `ollama serve` to setup ollama
-    * In terminal 2 -> `ollama run [model name]`, for example `ollama run llama2`
-        * The models can vary as you can create your own model. You can also view ollama's [library](https://ollama.ai/library) of models.
-    * This can also be done in [wsl](https://learn.microsoft.com/en-us/windows/wsl/install) for Windows machines.
-* You can now interact with the model you just ran (it might take a second to startup).
-    * Response time varies with processing power!
-
-## Project Setup
+## Environment Setup
 * Clone this repo using `git clone https://github.com/kevinthedang/discord-ollama.git` or just use [GitHub Desktop](https://desktop.github.com/) to clone the repo.
 * You will need a `.env` file in the root of the project directory with the bot's token. There is a `.env.sample` is provided for you as a reference for what environment variables.
     * For example, `CLIENT_TOKEN = [Bot Token]`
-
-## To Run (with Docker)
-* Follow this guide to setup [Docker](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04)
-    * If on Windows, download [Docker Desktop](https://docs.docker.com/desktop/install/windows-install/) to get the docker engine.
-* You will need a model in the container for this to work properly, on Docker Desktop go to the `Containers` tab, select the `ollama` container, and select `Exec` to run as root on your container. Now, run `ollama pull [model name]` to get your model.
-    * For Linux Servers, you need another shell to pull the model, or if you run `docker-compose build && docker-compose up -d`, then it will run in the background to keep your shell. Run `docker exec -it ollama bash` to get into the container and run the samme pull command above.
-* There is no need to install any npm packages for this, you just need to run `npm run start` to pull the containers and spin them up.
-* For cleaning up on Linux (or Windows), run the following commands:
-    * `docker-compose stop`
-    * `docker-compose rm`
-    * `docker ps` to check if containers have been removed.
-
-## To Run Locally (without Docker)
-* Run `npm install` to install the npm packages.
-* Now, you can run the bot by running `npm run client` which will build and run the decompiled typescript and run the setup for ollama.
-    * **IMPORTANT**: This must be ran in the wsl/Linux instance to work properly! Using Command Prompt/Powershell/Git Bash/etc. will not work on Windows (at least in my experience).
-    * Refer to the [resources](#resources) on what node version to use.
-* Open up a separate terminal/shell (you will need wsl for this if on windows) and run `ollama serve` to startup ollama.
-    * If you do not have a model, you will need to run `ollama pull [model name]` in a separate terminal to get it.
+* Please refer to the docs for bot setup. **NOTE**: These guides assume you already know how to setup a bot account for discord.
+    * [Local Machine Setup](./docs/setup-local.md)
+    * [Docker Setup for Servers and Local Machines](./docs/setup-docker.md)
+        * Local use is not recommended.
 
 ## Resources
 * [NodeJS](https://nodejs.org/en)

docker-compose.yml

@@ -8,6 +8,7 @@ services:
     build: ./                     # find docker file in designated path
     container_name: discord
     restart: always               # rebuild container always
+    image: discord/bot:0.2.0
     environment:
       CLIENT_TOKEN: ${CLIENT_TOKEN}
       GUILD_ID: ${GUILD_ID}
@@ -17,7 +18,8 @@ services:
       OLLAMA_IP: ${OLLAMA_IP}
       OLLAMA_PORT: ${OLLAMA_PORT}
     networks:
-      - ollama-net
+      ollama-net:
+        ipv4_address: ${DISCORD_IP}
     volumes: 
       - discord:/src/app          # docker will not make this for you, make it yourself
 
@@ -27,18 +29,24 @@ services:
     container_name: ollama
     restart: always
     networks:
-      - ollama-net
+      ollama-net:
+        ipv4_address: ${OLLAMA_IP}
+
     # runtime: nvidia             # use Nvidia Container Toolkit for GPU support
     # devices:
     #   - /dev/nvidia0
     volumes:
       - ollama:/root/.ollama
     ports: 
-      - 11434:11434
+      - ${OLLAMA_PORT}:${OLLAMA_PORT}
 
 networks:
   ollama-net:
     driver: bridge
+    ipam:
+      driver: default
+      config:
+        - subnet: ${SUBNET_ADDRESS}/16
 
 volumes:
   ollama:

docs/setup-docker.md

@@ -0,0 +1,31 @@
+## Docker Setup
+* Follow this guide to setup [Docker](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04)
+    * If on Windows, download [Docker Desktop](https://docs.docker.com/desktop/install/windows-install/) to get the docker engine.
+* Please also install [Docker Compose](https://docs.docker.com/compose/install/linux/) for easy running. If not, there are [scripts](#manual-run-with-docker) to set everything up.
+
+## To Run (with Docker and Docker Compose)
+* You will need a model in the container for this to work properly, on Docker Desktop go to the `Containers` tab, select the `ollama` container, and select `Exec` to run as root on your container. Now, run `ollama pull [model name]` to get your model.
+    * For Linux Servers, you need another shell to pull the model, or if you run `docker compose build && docker compose up -d`, then it will run in the background to keep your shell. Run `docker exec -it ollama bash` to get into the container and run the samme pull command above.
+* Otherwise, there is no need to install any npm packages for this, you just need to run `npm run start` to pull the containers and spin them up.
+* For cleaning up on Linux (or Windows), run the following commands:
+    * `docker compose stop`
+    * `docker compose rm`
+    * `docker ps` to check if containers have been removed.
+* You can also use `npm run clean` to clean up the containers and remove the network to address a possible `Address already in use` problem.
+
+## Manual Run (with Docker)
+* Run the following commands:
+    * `npm run docker:build`
+    * `npm run docker:ollama`
+    * `npm run docker:client`
+    * `docker ps` to see if the containers are there!
+        * Names should be **discord** and **ollama**.
+    * You can also just run `npm run docker:start` now for the above commands.
+* Clean-up:
+    * `docker ps` for the conatiner id's. Use `-a` flag as necessary.
+    * `docker rm -f discord && docker rm -f ollama` to remove the containers.
+        * `docker rm -f CONTAINER_ID` do for both containers if naming issues arise.
+    * `docker network rm ollama-net` removes the network.
+        * `docker network prune` will also work so long as the network is unused.
+* Remove Image:
+    * If you need to remove the image run `docker image rm IMAGE_ID`. You can get the image id by running `docker images`.

docs/setup-local.md

@@ -0,0 +1,19 @@
+## Ollama Setup
+* Go to Ollama's [Linux download page](https://ollama.ai/download/linux) and run the simple curl command they provide. The command should be `curl https://ollama.ai/install.sh | sh`.
+* Now the the following commands in separate terminals to test out how it works!
+    * In terminal 1 -> `ollama serve` to setup ollama
+    * In terminal 2 -> `ollama run [model name]`, for example `ollama run llama2`
+        * The models can vary as you can create your own model. You can also view ollama's [library](https://ollama.ai/library) of models.
+    * If there are any issues running ollama because of missing LLMs, run `ollama pull [model name]` as it will pull the model if Ollama has it in their library.
+    * This can also be done in [wsl](https://learn.microsoft.com/en-us/windows/wsl/install) for Windows machines.
+* You can now interact with the model you just ran (it might take a second to startup).
+    * Response time varies with processing power!
+
+## To Run Locally (without Docker)
+* Run `npm install` to install the npm packages.
+* Ensure that your [.env](../.env.sample) file's `OLLAMA_IP` is `127.0.0.1` to work properly.
+* Now, you can run the bot by running `npm run client` which will build and run the decompiled typescript and run the setup for ollama.
+    * **IMPORTANT**: This must be ran in the wsl/Linux instance to work properly! Using Command Prompt/Powershell/Git Bash/etc. will not work on Windows (at least in my experience).
+    * Refer to the [resources](../README.md#resources) on what node version to use.
+* Open up a separate terminal/shell (you will need wsl for this if on windows) and run `ollama serve` to startup ollama.
+    * If you do not have a model, you will need to run `ollama pull [model name]` in a separate terminal to get it.