Skip to content

Conversation

ggerganov
Copy link
Member

fix #11092

Repro:

/bin/llama-server -m ../models/codegemma-2b/ggml-model-f16.gguf 

curl -XPOST "localhost:8080/infill" -d '{"input_prefix": "1, ", "input_suffix": ", 5"}' -H "Content-Type: application/json" | jq

@ggerganov ggerganov requested a review from ngxson as a code owner January 6, 2025 12:47
@github-actions github-actions bot added examples python python script changes server labels Jan 6, 2025
@ggerganov ggerganov merged commit e6e7c75 into master Jan 6, 2025
51 checks passed
@ggerganov ggerganov deleted the gg/server-infill-fix-bos branch January 6, 2025 13:36
tinglou pushed a commit to tinglou/llama.cpp that referenced this pull request Feb 13, 2025
* server : fix extra BOS in infill endpoing

ggml-ci

* server : update infill tests
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Feb 26, 2025
* server : fix extra BOS in infill endpoing

ggml-ci

* server : update infill tests
mglambda pushed a commit to mglambda/llama.cpp that referenced this pull request Mar 8, 2025
* server : fix extra BOS in infill endpoing

ggml-ci

* server : update infill tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

examples python python script changes server

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misc. bug: server /infill endpoint incorrectly inserts <bos> token

2 participants