Allow send list of str for the Prompt on openai demo endpoint /v1/completions

[ironpinguin]

The langchain implementation sends the prompt as an array of strings to the /v1/completions endpoint.

With this change, it is possible to use a simple string or an array of strings to send the prompt.

If the prompt is an array, then we concatenate all strings to one string. And the following engine will work with both prompt data types.

This is a solution for #186

[zhuohan123]

Hi @ironpinguin! Thanks for the contribution! However, I believe this is not how OpenAI API behaves. Can you take a look at the example below?

import openai

completion = openai.Completion.create(
    model="text-davinci-003", prompt=["Say", "this", "is", "a", "test"], echo=True, n=1,
    stream=stream)

print(completion)

Output:

{
  "choices": [
    {
      "finish_reason": "length",
      "index": 0,
      "logprobs": null,
      "text": "Say(\"T\");\n\t\t\tSingSay(\"S\");\n\t\t\t"
    },
    {
      "finish_reason": "length",
      "index": 1,
      "logprobs": null,
      "text": "this->get_setting_value('show_tax_totals_in"
    },
    {
      "finish_reason": "length",
      "index": 2,
      "logprobs": null,
      "text": "is_eof()) {\n\t\t<*ddc>\n\t\t"
    },
    {
      "finish_reason": "length",
      "index": 3,
      "logprobs": null,
      "text": "aient \u00e0 tous moment (avec leur million de pi\u00e8ces en"
    },
    {
      "finish_reason": "length",
      "index": 4,
      "logprobs": null,
      "text": "test.mp3','rb') as f: #rb \ufffd\ufffd\ufffd\ufffd\ufffd\ufffd \ufffd\ufffd\ufffd"
    }
  ],
  "created": 1688142843,
  "id": "cmpl-7XBMJLEFOhWfBg5ngxoMBiD5vMSzh",
  "model": "text-davinci-003",
  "object": "text_completion",
  "usage": {
    "completion_tokens": 79,
    "prompt_tokens": 5,
    "total_tokens": 84
  }
}

OpenAI API is treating the strings in the list as separate prompts.

As a temp fix, when the request.prompt is a list, can you only proceed when its length is 1?

[zhuohan123]

LGTM! Thank you for your contribution!

[XBeg9]

Hi, just want to be sure that it's on my side and it's not a regression. Trying to use langchain with vllm and I am getting exactly this problem:

from langchain.llms import VLLMOpenAI

llm = VLLMOpenAI(
    openai_api_key="EMPTY",
    openai_api_base="http://localhost:8000/v1",
    model_name="TheBloke/Llama-2-70B-chat-AWQ",
    model_kwargs={"stop": ["."]},
)
print(llm("Rome is"))

with response Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIError: Invalid response object from API: '{"detail":[{"loc":["body","prompt"],"msg":"str type expected","type":"type_error.str"}]}' (HTTP response code was 422).