[Feature]: Support openai responses API interface

[guunergooner]

🚀 The feature, motivation and pitch

OpenAI has released a new Responses API, but vLLM does not currently support it.
We request that vLLM adds compatibility with this API to stay in sync with OpenAI's updates.
Supporting the Responses API will enhance vLLM's utility and competitiveness.
For more details, see the OpenAI documentation

• OpenAI Responses api reference
  https://platform.openai.com/docs/api-reference/responses

• OpenAI Responses vs. Chat Completions
  https://platform.openai.com/docs/guides/responses-vs-chat-completions

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[JaimeArboleda]

I would be super interested as well, but I think the problem is that responses API is stateful, which breaks a lot of assumptions in terms of vLLM architecture, if I am not mistaken... Another important difference is the one related to files and vector stores. I do not see how a project like vLLM, aimed only at solving the LLM inference problem in an efficient an scalable way, could handle that.

[aarnphm]

I think responses API is strictly frontend, so it is possible to make it stateful to support this API.

It won't affect any core logics (core logics should still be stateless).

fwiw I think it would be even better to have a frontend plugins for this, as a part of vllm community project if we don't want to add it to core vllm.

[jhuntbach-bc]

The state can still be managed by the calling client (outside of vLLM) in the same way as the chat completions endpoint
https://platform.openai.com/docs/guides/conversation-state?api-mode=responses#manually-manage-conversation-state

vLLM could implement the responses API, and just ignore the previous_response_id parameters and similar, to keep a stateless architecture.

[jhuntbach-bc]

Similarly, for files, we could just ignore file_id params, and enforce that the file should be provided in each request

https://platform.openai.com/docs/guides/pdf-files?api-mode=responses#base64-encoded-files

[rustonaut]

From what I can tell /responses seem to be basically a more clean redesign of /chat/completions with some added features related to more conveniently accessing conversation state (i.e. previous_response_id , file_id , store parameter and the ItemReference InputItem variant) as well as some more build-in tools.

I think it it makes a lot of sense to provide such an API which is compatible except not supporting this features.

I see two reasons for it:

• OpenAI has been recommending /responses over /chat/completions for new projects (in general not "just if you need one of the new features") so it's likely for applications to want compatibility with this API interface

• For applications build on top of vLLM which do provide document preprocessing, session storage etc. (but not necessary in the way OpenAI does) it can be interesting to expose a (at lest somewhat) /responses compatible API. And being able to just take the /responses API resolve all state related parts and do some domain specific transformations/enrichment and then forward it to vLLM would be very nice.

but also some drawbacks:

• If we already have /responses people will ask for also adding support for the conversation state related APIs. I have played a bit around with ways vLLM could bet setup to automatically call out to tools, file preprocessing, conversation state storage etc. But due to the vastly different requirements for handling user data (and in turn conversation state) I think it's best to always resolve it in a more use-case specific layer on top of vLLM 🤔, through fully 100% sure.

[OriNachum]

This is now a requirement, as Codex requires that endpoint.
Ollama already has a PR to support it.

For Codex to support you need the "store: true" feature supported (To enable reasoning)

I wrote a wrapper responses api -> chat completions.
It doesn't write code properly, it needs the "store: true" for reasoning
Following readme here

I see:

Zero Data Retention (ZDR) Organization Limitation
Note: Codex CLI does not currently support OpenAI organizations with Zero Data Retention (ZDR) enabled.

If your OpenAI organization has Zero Data Retention enabled, you may encounter errors such as:

OpenAI rejected the request. Error details: Status: 400, Code: unsupported_parameter, Type: invalid_request_error, Message: 400 Previous response cannot be used for this organization due to Zero Data Retention.
Why?

Codex CLI relies on the Responses API with store:true to enable internal reasoning steps.
As noted in the docs, the Responses API requires a 30-day retention period by default, or when the store parameter is set to true.
ZDR organizations cannot use store:true, so requests will fail.
What can I do?

If you are part of a ZDR organization, Codex CLI will not work until support is added.
We are tracking this limitation and will update the documentation if support becomes available.

[OriNachum]

@rustonaut, I wrote on a middleware server to bridge responses api to chat completions endpoint.
It manages the events, so you can use that with Vllm and run Codex

Pypi package: https://pypi.org/project/open-responses-server/

Git repo: https://github.com/teabranch/open-responses-server

Edit: changed name from OpenAI to open, to avoid brand name