feat(liteLLM): Add support for Azure OpenAI, Palm, Claude-2, Llama2, CodeLlama (100+LLMs)

[ishaan-jaff]

Description

This PR adds support for the above mentioned LLMs using LiteLLM https://github.com/BerriAI/litellm/
Example:
docs on langchain integration https://python.langchain.com/docs/integrations/chat/litellm

ChatLiteLLM(model="gpt-3.5-turbo")
ChatLiteLLM(model="claude-2", temperature=0.3)
ChatLiteLLM(model="command-nightly")
ChatLiteLLM(model="replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1")

In addition LiteLLM client allows you to n [Experimental + Optional Features]:

• A/B test LLMs in production
• Dynamically control each LLMs prompt, temperature, top_k etc in our UI (no need to re-deploy code)
• Logging to view input/outputs for each LLM

Here's a link to a live demo of litellm client: https://admin.litellm.ai/

Please include a summary of the changes and the related issue. Please also include relevant motivation and context.

Checklist before requesting a review

Please delete options that are not relevant.

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented hard-to-understand areas
• I have ideally added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged

Screenshots (if appropriate):

[ishaan-jaff]

@StanGirard can you please take a look at this PR when possible ? Happy to add more docs/tests if this initial commit looks good😊

[StanGirard]

Looks really cool. I don’t have my pc with me currently but here are a few notes:

• don’t you need to install litellm package in the requirements.txt ?
• how can I deploy my own admin ?

[aadityamundhalia]

@ishaan-jaff i think instead of replacing the current import you should add .env variable and with if statement allow both openai as well as litellm

[StanGirard]

@ishaan-jaff i think instead of replacing the current import you should add .env variable and with if statement allow both openai as well as litellm

The goal is to replace openai with a project that allows anyone :)

[ishaan-jaff]

langchain 0.0.281 includes the latest version of ChatLiteLLM too

[ishaan-jaff]

• don’t you need to install litellm package in the requirements.txt ?

@StanGirard done, good catch. Also bumped the version of langchain to include the latest ChatLiteLLM fixes https://github.com/langchain-ai/langchain/releases

• how can I deploy my own admin ?

This is an experimental feature using completion_with_split_tests. Happy to add docs to quivr about how to do this too, here's a link to our docs: https://docs.litellm.ai/docs/tutorials/ab_test_llms

Just want to re-iterate that the dashboard is experimental and our focus is entirely on making LiteLLM really good at I/O

[StanGirard]

@gozineb looks good to me. Can you check that everything works ? Especially with the version update of langchain ?

Should only need to test:
Uploading
Chat with a brain and unplug.

[mamadoudicko]

Do we still needlitellm import ?

[StanGirard]

I think so :) langchain is just a wrapper. It doesn't come with the packages.

[mamadoudicko]

Sounds good

[gozineb]

@StanGirard looks good to me ! Merging this as soon as the checks pass !

[NormTurtle]

what about PALM
which has free API from google?

[ishaan-jaff]

we support palm too https://docs.litellm.ai/docs/providers