[Provider] Predibase

[roh26it]

Add Predibase as an AI provider integration.

They allow easy use of adapters

[VisargD]

In predibase, they allow using adapters for each request. The structure of adapter is simple key value pair that is sent in the body: "adapter_id": "<finetuned_model_repo_name>/<finetuned_model_version>"

I was thinking that would it make sense to have adapter as a tool in gateway request body just like function, code_interpreter and retrieval are a tool? This way the request still remains OpenAI compatible. Something like this:

"tools": [
        {
            "type": "adapter",
            "adapter_id": "<finetuned_model_repo_name>/<finetuned_model_version>"
        }
    ]

Just wanted to have some feedback on this. This might also cause friction for someone switching from Predibase APIs to Portkey Gateway APIs. But this looks like a more suitable option as it aligns with the idea of unified interface and can also be extended in future.

The other easy way is to simply allow a new field called adapter_id which will be forwarded to Predibase as it is.

[tgaddair]

Hello, with LoRAX / Predibase's OpenAI comptible API, it should hopefully be fairly straightforward to integrate:

https://predibase.github.io/lorax/guides/openai_api/

The model param here can be used to specify the adapter_id.

For measuring token consumption, LoRAX / Predibase returns the following information in the response headers:

• x-prompt-tokens: number of tokens in the input prompt
• x-generated-tokens: number of tokens generated by the model
• x-total-tokens promot.+ generated tokens

Predibase's per-token pricing for serverless LLMs can be found here: https://predibase.com/pricing

[flexchar]

I'm exploring Predibase for my adapters, I might swing by this too.