Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LM studio not ignoring tool call blocks within <think> tags #453

Open
ghhwer opened this issue Feb 21, 2025 · 1 comment
Open

LM studio not ignoring tool call blocks within <think> tags #453

ghhwer opened this issue Feb 21, 2025 · 1 comment
Assignees

Comments

@ghhwer
Copy link

ghhwer commented Feb 21, 2025

Which version of LM Studio?
Example: LM Studio 0.3.10 (Build 2)

Which operating system?
Windows

What is the bug?
LLMs that use think tags like deepseek-r1-distill-llama-8b@q8_0 sometimes can use [TOOL_REQUEST] blocks within their tags, however it does not seem to be the LLM's intent to call [TOOL_REQUEST] it's just "reasoning" about their input.

This leads to incorrect outputs or repeated calls.

Screenshots

Image You can see the "red" markings that show the LLM calling [TOOL_REQUEST] blocks.

Blue shows the LM Studio engine calling the tool 3 times.
Yellow shows the result of the LLM straying from the objective.

Logs
Not needed

To Reproduce
Steps to reproduce the behavior:

  1. Have a deepseek-r1-distill-llama-8b@q8_0 model set up with a modestly complex tool.
  2. Have it reason on the response.
  3. You will observe the behavior.

Proposition to solve
LM Studio should ignore TOOL REQUESTS within think tags to avoid this issue.

@ghhwer
Copy link
Author

ghhwer commented Feb 23, 2025

Also to add more context for anyone facing the issue, if you are expecting the model to only ever call the tool once, you can always consider the last element in tool call order.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants