Merlinite model needs "chat_format" set to "openchat"

[MichaelClifford]

Some models in our list have a different chat format. The environment variable CHAT_FORMAT needs to be set when starting the playground or model server image with these models. Without this parameter set correctly the models will not return valid responses.

https://github.com/containers/ai-lab-recipes/blob/38f66d8fa8b62ba63bb469083f0c32acf1793955/model_servers/llamacpp_python/src/run.sh#L8C5-L8C213

The affected models are:

• Merlinite
• Cerebrum
• dragon-mistral

All three requires CHAT_FORMAT="openchat"

[axel7083]

The image we are using ghcr.io/containers/podman-desktop-extension-ai-lab-playground-images/ai-lab-playground-chat:0.2.0 is not up to date.

$: podman run --entrypoint=/bin/bash -it ghcr.io/containers/podman-desktop-extension-ai-lab-playground-images/ai-lab-playground-chat:0.2.0 -c "cat /home/default/run.sh"
#!/bin/bash
#
# Copyright (C) 2024 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0
python -m llama_cpp.server --model ${MODEL_PATH} --host ${HOST:=0.0.0.0} --port ${PORT:=8000} --n_gpu_layers 0