Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gaudi teirerank is in crash loop after Helm install #483

Open
eero-t opened this issue Oct 17, 2024 · 0 comments
Open

Gaudi teirerank is in crash loop after Helm install #483

eero-t opened this issue Oct 17, 2024 · 0 comments

Comments

@eero-t
Copy link
Contributor

eero-t commented Oct 17, 2024

teirerank ends in in crash loop with git HEAD when its Gaudi version (#475) is used:

2024-10-17T17:31:58.855352Z  INFO text_embeddings_backend_python::management: backends/python/src/management.rs:136: Python backend process terminated
Error: Error when doing warmup

Caused by:
    Could not start backend: max_warmup_length (1024) exceeds model's max_input_length (512), you can modify this value adding `-e MAX_WARMUP_SEQUENCE_LENGTH=<new_warmup_length>` to your Docker run command

Does this not happen with tei CPU version?

Manually adding same MAX_WARMUP_SEQUENCE_LENGTH=512 to teirerank configMap that tei configMap has, fixes the issue.

=> Fixing #472 would fix also this discrepancy.
@eero-t eero-t mentioned this issue Oct 18, 2024
Open
1 task
eero-t added a commit to eero-t/GenAIInfra that referenced this issue Oct 18, 2024
@eero-t
Fix: use MAX_WARMUP_SEQUENCE_LENGTH for both tei & teirerank
c92afa7 
Max input length applies to both, so teirerank needs also max warmup length.

Fixes: opea-project#483

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@eero-t