-
Notifications
You must be signed in to change notification settings - Fork 477
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Supporting Cline requests style. #2659
Comments
I have no clue about cline, to be honest, this is my first time hearing of it. But welcome to contribute on this feature. |
Cline is a VS code extension, previous named Claude Dev. I think it would be a lot of work since every LLM model has it's own |
Does Cline has its own API router like openai ones /v1, if so, we can tell the message is from Cline, and we can write a function to convert the messages. |
Cline is design to work with Anthropic Claude 3.5 Sonnet, it should be OpenAI API compatible. I think it just a client that construct requests and then send to API. It's hard to tell whether the request is from Cline. The request constructed does not contain any information says it's from Cline. |
That would a bit more nasty, we have to analyze the content of messages, if it contains the structure like the example, we convert it to flattened normal openai messages. |
Yes, and will bring extra burden to the server on each request. Do you think it's worthy? Are there any other clients that send similar request? |
Yeah, it costs some, a simple way is just pass the messages to models, if TypeError raised, we then check and covert, IMO, that would be better. |
liteLLM is using this strategy.
If the requests content are all |
OK, for messages we do the conversion I think it's OK. |
This issue is stale because it has been open for 7 days with no activity. |
@gahoo Do you have interest to contribute this feature? |
Thanks @hwzhuhao , nice work, I prefer the method 2 which is more general. Willing to hear your opnions. |
I also agree to use method 2. The chat-template is usually provided by the model provider, so it is not suitable for modifications. |
I modified the code in the Docker image using method 2 above, but it didn't work |
Feature request / 功能建议
Cline
requests are quite different in thecontent
section which might be alist
instead ofstr
.So the default
chat_template
might cause the following error:Motivation / 动机
It would be great that Cline or other similar tools can use Xinference's API.
Similar issues happened in DeepSeek API, and seems fixed. see cline/cline#230 .
Your contribution / 您的贡献
None
The text was updated successfully, but these errors were encountered: