GROQ Provider: finish_reason returned as integer causes incompatibility with go-openai client

[elliot14A]

What Happened?

When using the GROQ provider through Portkey API with go-openai client, the streaming response fails due to a type mismatch. GROQ returns the finish_reason field as an integer (0), but the go-openai client expects it to be a string type ("stop"). This causes the following error:

json: cannot unmarshal number into Go struct field ChatCompletionStreamChoice.choices.finish_reason of type openai.FinishReason

Example GROQ response showing the integer finish_reason:

data: {"id":"chatcmpl-a72900d4-d3f7-4c83-afef-3bd60976cfa1","object":"chat.completion.chunk","created":1733195769,"model":"llama-3.2-90b-vision-preview","provider":"groq","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":0}],"usage":{"prompt_tokens":41,"completion_tokens":55,"total_tokens":96}}

sending the finish_reason as integer might cause issues with other clients/libraries as well not just go openai client.

client library i used: github.com/sashabaranov/go-openai version: v1.36.0

Would you like me to submit a PR to fix this issue?

What Should Have Happened?

The GROQ provider should have returned the finish_reason as a string value in the streaming response, maintaining compatibility with the go-openai client's expected data structure. The response should have looked like this:

data: {"id":"chatcmpl-AaDke27DNU3bWfnHf1GGqAyPiY0NY","object":"chat.completion.chunk","created":1733196032,"model":"gpt-4o-2024-08-06","system_fingerprint":"fp_831e067d82","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]}

Relevant Code Snippet

GROQ Request:

curl -N https://api.portkey.ai/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "x-portkey-provider: groq" \
      -d '{
    "model": "llama-3.2-90b-vision-preview",
    "messages": [{"role": "user", "content": "Hello, how are you?"}],
    "stream": true,
    "temperature": 0.7,
    "max_tokens": 512
  }'

OPENAI Request:

curl -N https://api.portkey.ai/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "x-portkey-provider: openai" \
     -d '{
   "model": "gpt-4o",
   "messages": [{"role": "user", "content": "Hello, how are you?"}],
   "stream": true,
   "temperature": 0.7,
   "max_tokens": 512
 }'

Your Twitter/LinkedIn

No response

[narengogi]

@elliot14A we consciously chose to not transform the values of the fields to avoid loss in context.
While we map fields like stop_reason, reason etc to finish_reason, we don't transform the value inside.

We attempted doing this before, here's the relevant discussion:
#468 (comment)
cc: @VisargD

[elliot14A]

@narengogi so what would a possible work around for my case then ?

[narengogi]

@elliot14A You could modify the response type of openai client package, or define your own types while using Portkey HTTP endpoint (https://api.portkey.ai/v1) for making the requests

[elliot14A]

I just crossed checked the groq api

data: {"id":"chatcmpl-ec11a3f2-7433-4517-9315-219552760f92","object":"chat.completion.chunk","created":1733208790,"model":"llama-3.2-90b-vision-preview","system_fingerprint":"fp_9c2a937c92","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}],"x_groq":{"id":"req_01je5mpm3bfd6anak07x2dae5k","usage":{"queue_time":0.010089255000000002,"prompt_tokens":43,"prompt_time":0.004748735,"completion_tokens":455,"completion_time":1.70097023,"total_tokens":498,"total_time":1.705718965}}}

the finish_reason sent in response here is string.

[narengogi]

oh, this is a bug then, I'll prioritize fixing this, thanks for the report @elliot14A!

[deshetti]

@narengogi thanks for the prompt response. Do you have an ETA on when the fix will be implemented?

[elliot14A]

@narengogi any update ?

[narengogi]

raised PR @elliot14A