Implement /v1/chat/completions endpoint for CPU mode

[johannesploetner]

Describe your changes

The /v1/chat/completions endpoint was not implemented in gpt4all-api (only returned an "Echo" of the original message, as mentioned in #1700). This PR implements the endpoint for CPU mode and adds an appropriate test.

Issue ticket number and link

#1700

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• I have added thorough documentation for my code.
• I have tagged PR with relevant project labels. I acknowledge that a PR without labels may be dismissed.
• If this PR addresses a bug, I have provided both a screenshot/video of the original bug and the working solution.

Demo

try the openai.ChatCompletion.create() function (as described here: https://cookbook.openai.com/examples/how_to_format_inputs_to_chatgpt_models)

import openai

openai.api_base="http://localhost:4891/v1"
openai.api_key="ABCDEFG"

stream = openai.ChatCompletion.create(
    model="mistral-7b-openorca.Q4_0.gguf",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Knock knock."},
        {"role": "assistant", "content": "Who's there?"},
        {"role": "user", "content": "Orange."},
        ],
       temperature=0
)
print(stream.choices[0].message.content)

Before the PR, we would just get an "Echo: " of the last message. After the PR, the we actually get a result.

Steps to Reproduce

• see test_chat_completion() in gpt4all_api/app/tests/test_endpoints.py

Notes

• GPU support will need to be added later. Worth merging anyway IMHO, as it actually adds functionality.

[cebtenzzre]

Why is ChatML hard-coded here? Normally GPT4All models have a customizable prompt template that defaults to the value in models2.json.

[johannesploetner]

This is done tue to the very specifics of OpenAI's API. OpenAI's ChatCompletions endpoint can receive an array of messages containing the system message, and past user questions/statements and the assistants replies (see https://platform.openai.com/docs/api-reference/chat/create and https://cookbook.openai.com/examples/how_to_format_inputs_to_chatgpt_models):

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Knock knock."},
            {"role": "assistant", "content": "Who's there?"},
            {"role": "user", "content": "Orange."},
    ]
  }'

Goal was here to parse the past messages and to provide the LLM with the complete history. From the above example, I am creating a result like this:

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Knock knock.<|im_end|>
<|im_start|>assistant
Who's there?<|im_end|>
<|im_start|>user
Orange.<|im_end|>
<|im_start|>assistant

IMHO we would a different representation of prompt templates in model2.json to be able to reliably parse and use those for this specific use case... Until then, I decided to hardcode ChatML.