fix: adding chatformat to use for inference servers

[axel7083]

What does this PR do?

This PR is adding the optional chatformat to the model object.

Need containers/podman-desktop-extension-ai-lab-playground-images#11 to be merged, and a new version published of the image to be working.

Fixing the playgrounds for the following models

• Merlinite
• Cerebrum
• dragon-mistral

Screenshot / video of UI

What issues does this PR fix or reference?

Fixes #825

How to test this PR?

• unit tests has been provided

[jeffmaury]

As a general comment I think this is too specific for chat models. We already have the plan to add a type attribute to models (chat, vision,...) so I think it would be better to rename the field to something more general purpose (format ?). And the env variable should be renamed accordingly

[jeffmaury]

Was thinking intensively about this during the night: found a more generic way to handle that.
Basically we want to pass model attributes to the containers so let's add an attributes field to the model catalog and this will be a record of values. When launching the container, pass each value of this record as a env variable named MODEL_key where key is the uppercase transcription eg if the key is chatformat then the env var would be MODEL_CHATFORMAT.
This would requires a small change on the recipes but allows for better evolution in the future if new attributes are added.
WDYT ?

[benoitf]

freeform is more extensible 👍

about keys is it using camelCase ?

chatFormat -> CHAT_FORMAT ?

[axel7083]

MODEL_

I like this solution but I am not sure about the necessity of the MODEL_ prefix ?

[benoitf]

"properties" : {
  "chatFormat": "openchat"
}

?

[jeffmaury]

MODEL_

I like this solution but I am not sure about the necessity of the MODEL_ prefix ?

The MODEL_ prefix just to standardize on what is provider by AI Lab as we already provider MODEL_PATH and as it's related to the model

[jeffmaury]

MODEL_

I like this solution but I am not sure about the necessity of the MODEL_ prefix ?

The MODEL_ prefix just to standardize on what is provided by AI Lab as we already provide MODEL_PATH and as it's related to the model

[axel7083]

"properties" : {
  "chatFormat": "openchat"
}

?

We are using the labels wording in several places, might want to stick with that ?

podman-desktop-extension-ai-lab/packages/shared/src/models/ITask.ts

Line 27 in 1882d0c

labels?: { [id: string]: string };

podman-desktop-extension-ai-lab/packages/shared/src/models/ILocalRepository.ts

Line 4 in 5dd413e

labels: { [id: string]: string };

podman-desktop-extension-ai-lab/packages/shared/src/models/InferenceServerConfig.ts

Line 38 in 039b7ca

labels: { [id: string]: string };

[jeffmaury]

"properties" : {
  "chatFormat": "openchat"
}

?

We are using the labels wording in several places, might want to stick with that ?

podman-desktop-extension-ai-lab/packages/shared/src/models/ITask.ts

Line 27 in 1882d0c

labels?: { [id: string]: string };

podman-desktop-extension-ai-lab/packages/shared/src/models/ILocalRepository.ts

Line 4 in 5dd413e

labels: { [id: string]: string };

podman-desktop-extension-ai-lab/packages/shared/src/models/InferenceServerConfig.ts

Line 38 in 039b7ca

labels: { [id: string]: string };

labels are intended for display I think this is more internal here so I would be more for attributes or properties

[axel7083]

labels are intended for display I think this is more internal here so I would be more for attributes or properties
Go for properties then !

The prefix would not work with containers/podman-desktop-extension-ai-lab-playground-images#11 for now, might want to not having it for now ?

[axel7083]

"properties" : {
  "chatFormat": "openchat"
}

chat_format is easier to transform to CHAT_FORMAT than chatFormat, we jsut have to use uppercase() for the first one

[benoitf]

I think that I expect camelCase in a json file also about easier I'm not sure it would be much harder anyway

'chatFormat'.replace(/[A-Z]/g, m => `_${m}`).toUpperCase() => CHAT_FORMAT

[axel7083]

I think that I expect camelCase in a json file also about easier I'm not sure it would be much harder anyway

No problem then

[jeffmaury]

labels are intended for display I think this is more internal here so I would be more for attributes or properties
Go for properties then !

The prefix would not work with containers/podman-desktop-extension-ai-lab-playground-images#11 for now, might want to not having it for now ?

I would favor (and not delay) the prefix is the ai-recipes team is ok with this change, starting the discussion on their channel

[axel7083]

labels are intended for display I think this is more internal here so I would be more for attributes or properties
Go for properties then !

The prefix would not work with containers/podman-desktop-extension-ai-lab-playground-images#11 for now, might want to not having it for now ?

I would favor (and not delay) the prefix is the ai-recipes team is ok with this change, starting the discussion on their channel

✅ done

[benoitf]

so here we need to update https://github.com/containers/podman-desktop-extension-ai-lab-playground-images/pull/11/files to use MODEL_ env var

[jeffmaury]

LGTM. Recipe repo has been updated so we should be good

[benoitf]

before merging,

podman-desktop-extension-ai-lab/packages/backend/src/utils/inferenceUtils.ts

Line 37 in 1f4223a

'ghcr.io/containers/podman-desktop-extension-ai-lab-playground-images/ai-lab-playground-chat:0.2.0';

need to be updated and a new release should happen in the https://github.com/containers/podman-desktop-extension-ai-lab-playground-images repo

[jeffmaury]

LGTM