You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
When the inference engine returns an error response that is not of type openai.ErrorResponse, a decode error occurs and it is returned in openai.RequestError, leading to the loss of the original error message from the inference server. Currently not all inference engines for example TGI and NIMs return error struct like openai.ErrorResponse, they follow different error struct so the actual error is lost through this go-openai client.
Describe the solution you'd like
Introduce a new field called Body of type []byte in the openai.RequestError struct. This field stores the byte slice of the error response body received from the inference engine when the error response does not conform to the openai.ErrorResponse type. This enhancement allows to retain the original error message. I have raised PR for this change #873. Now that the byte slice of the error response body is included, clients using this library can unmarshal it on their end to obtain the correct error message.
The text was updated successfully, but these errors were encountered:
With the ever-evolving space with many prominent runtimes like vLLM, Hugging Face TGI, Nvidia NIM supports the OpenAI protocol. Currently, error handling has been a hassle; with this support, the consumer can decide how to handle the exceptions gracefully.
Is your feature request related to a problem? Please describe.
When the inference engine returns an error response that is not of type openai.ErrorResponse, a decode error occurs and it is returned in openai.RequestError, leading to the loss of the original error message from the inference server. Currently not all inference engines for example TGI and NIMs return error struct like openai.ErrorResponse, they follow different error struct so the actual error is lost through this go-openai client.
Describe the solution you'd like
Introduce a new field called Body of type []byte in the openai.RequestError struct. This field stores the byte slice of the error response body received from the inference engine when the error response does not conform to the openai.ErrorResponse type. This enhancement allows to retain the original error message. I have raised PR for this change #873. Now that the byte slice of the error response body is included, clients using this library can unmarshal it on their end to obtain the correct error message.
The text was updated successfully, but these errors were encountered: