[FEAT]: Add pixtral vision capabilities

[nekopep]

What would you like to see?

Currenlty we are able to select new MistralAi Pixtral 12b model.

This model is a multimodal model specialized in image analysis. In latest AnythingLLM 1.2.2 we can upload image but we need the send the image to pixtral in base64 in the request.

messages = [
# Define the messages for the chat
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "What's in this image?"
            },
            {
                "type": "image_url"
                "image_url": f"data:image/jpeg;base64,{base64_image}"
            }
        ]
    }
]

It can be used to transcribe image (OCR) see example in image below. You can even compare two images.

I could be very cool to add this feature to anythingLLM :)

Doc about the feature:
https://docs.mistral.ai/capabilities/vision/#passing-a-base64-encoded-image

[timothycarambat]

Are you using the "Mistral" LLM provider using that uses their hosted API service? I am presuming so

[nekopep]

Yes, I'm using AnythingLLM as a "proxy" to diverse LLM provider (currently Mistral and ChatGPT)
Thanks for the merge ;)

[nekopep]

Hi timothy works perfectly! Thks!