Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return error response body when error is not of type openai.ErrorResponse #874

Open
AyushSawant18588 opened this issue Oct 10, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@AyushSawant18588
Copy link
Contributor

Is your feature request related to a problem? Please describe.
When the inference engine returns an error response that is not of type openai.ErrorResponse, a decode error occurs and it is returned in openai.RequestError, leading to the loss of the original error message from the inference server. Currently not all inference engines for example TGI and NIMs return error struct like openai.ErrorResponse, they follow different error struct so the actual error is lost through this go-openai client.

Describe the solution you'd like
Introduce a new field called Body of type []byte in the openai.RequestError struct. This field stores the byte slice of the error response body received from the inference engine when the error response does not conform to the openai.ErrorResponse type. This enhancement allows to retain the original error message. I have raised PR for this change #873. Now that the byte slice of the error response body is included, clients using this library can unmarshal it on their end to obtain the correct error message.

@gavrissh
Copy link

gavrissh commented Oct 10, 2024

With the ever-evolving space with many prominent runtimes like vLLM, Hugging Face TGI, Nvidia NIM supports the OpenAI protocol. Currently, error handling has been a hassle; with this support, the consumer can decide how to handle the exceptions gracefully.

Looking for it to be prioritized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants