[DRAFT] feat: support deepseek-reasoner

[LastRemote]

Related Issues

• fixes Support for o1-like reasoning model (LRMs) #8760

Proposed Changes:

Added a reasoning_content field for both ChatMessage and StreamingChunk

How did you test it?

This is not thoroughly tested and should stay as a draft PR.

Notes for the reviewer

Hello, this is a draft PR that adds support for reasoning models, with untested sample code to support deepseek-reasoner style APIs in OpenAIChatGenerator. I will be away for the rest of the week (responses might be delayed), but feel free to take over.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• [MISSING] I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I documented my code
• [MISSING] I ran pre-commit hooks and fixed any issue

[LastRemote]

Completed E2E test with real deepseek-reasoner API.

Additional notes: deepseek API has undergoing issues due to recent attacks and the generator would fail ungracefully. We might need to add more try-except clauses here and there.

[vblagoje]

@LastRemote is this some typo here? Why have the same nested if?

[LastRemote]

Yes, these are some redundancies. I rushed the code before the vacation without taking a deeper thought, sorry about that. Will fix and add appropriate testing/documentation.

[vblagoje]

@LastRemote a short reno note helps us make really nice release notes. Would you please add one to this PR?
There are a couple of issues with docstrings and linting, you can run these checks prior to committing by running:

hatch run check 
hatch run test:lint
hatch run test:types 

Looking forward to these updates and let's involve @anakin87 for a final check as well

[LastRemote]

@LastRemote a short reno note helps us make really nice release notes. Would you please add one to this PR? There are a couple of issues with docstrings and linting, you can run these checks prior to committing by running:

hatch run check 
hatch run test:lint
hatch run test:types 

Looking forward to these updates and let's involve @anakin87 for a final check as well

For sure. It was meant to be an untested sketch so I kind of rushed it before taking the vacation. Definitely needs some further polishing here and there.

[anakin87]

Sorry for not participating in the discussion...
I need some time to investigate. I hope to be able to do so soon.

[anakin87]

I finally found some time to look at this.

The primary goal of the changes introduced in this PR is supporting deepseek-reasoner, following their API reference.

I am a bit hesitant because I have tried DeepSeek R1 through different inference providers (Together.ai, DeepInfra) and I see that, in different ways, they put the thinking part in the textual content of the message.
Speaking of reasoning models in general, OpenAI does not yet officially support reasoning traces in its API, Google puts the thinking in the text... There does not seem to exist a standard yet.

In any case, we are not supporting the "official" DeepSeek API at the moment, so let's talk/discuss this topic @vblagoje.

[LastRemote]

@anakin87 Sorry for the late reply. I was a bit too busy these weeks.

There does not seem to exist a standard yet.

Yes. I expect this to converge at some point, but the main scope of this PR is to update ChatMessage structure in favor of reasoning context. We can discard the changes to OpenAIChatGenerator, but I see minimal harm taking an extra step here. As I said in #8760, my goal is to support something that works and we can always extend/switch to other formats should they become more popular.

Btw, my expectation is that the raw format (with <think> tokens) might only be a temporary solution since it is indistinguishable in some cases where the model generates <think> strings in its reasoning/response.

[vblagoje]

So in conclusion @LastRemote and @anakin87 we can add reasoning traces directly to text portion of the ChatMessage until some sort of a standard appears? And also have it include these think tags as users can then remove them if needed via simple regex?

[anakin87]

@vblagoje I was thinking something similar: for now, do not change ChatMessage, but only OpenAIChatGenerator to allow this use case.

[LastRemote]

My idea is quite the contrary: update ChatMessage to explicitly include reasoning content since it is clear that this becomes a trend, and potentially hold changes in OpenAIChatGenerator for a more well-respected reasoner API.

[vblagoje]

My idea is quite the contrary: update ChatMessage to explicitly include reasoning content since it is clear that this becomes a trend, and potentially hold changes in OpenAIChatGenerator for a more well-respected reasoner API.

Yes, fair point @LastRemote - let us sync internally and we'll move this one forward soon.

[vblagoje]

@LastRemote for the time being, after an internal discussion, we have decided that it is okay to make a small modification to OpenAIChatGenerator to include thinking in the text.
We prefer not to modify the ChatMessage (which is an important abstraction, used throughout the framework) until there is a shared standard between providers.
That said, it should be fairly easy to retrieve the thinking with simple post-processing of the text.

[LastRemote]

@vblagoje Thanks for the reply. It is understandable, although I would still prefer the other way.

I will need to re-evaluate what should be done to modify OpenAIChatGenerator without changing the structure of ChatMessage, likely in a lower priority. IMO there is not much to do since the only difference is that reasoning_content now becomes a part of ChatMessage.text. As long as I am concerned right now, sticking the reasoning content to the final response content doesn't look appealing.

P.S. As I mentioned earlier, I am currently using a heavily modified ChatGenerator component in my pipelines (with a customized ChatMessage dataclass), and that one already got full support for reasoning tokens (and multimodal content), so the current issue is not blocking me. But please let me know when you are ready to officially support reasoning tokens.