Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ci-auto] remove "--enforce-eager" for better vLLM perf #631

Closed
daisy-ycguo opened this issue Dec 10, 2024 · 0 comments · Fixed by #610
Closed

[ci-auto] remove "--enforce-eager" for better vLLM perf #631

daisy-ycguo opened this issue Dec 10, 2024 · 0 comments · Fixed by #610
Assignees
@lianhao
Labels
helm

Comments

@daisy-ycguo
Copy link
Contributor

GenAIExample ChatQnA compose.yaml got changed

Below files are changed in this commit

  • ChatQnA/docker_compose/intel/hpu/gaudi/compose_vllm.yaml

Please verify if the helm charts and manifests need to be changed accordingly.

This issue was created automatically by CI.
@lianhao lianhao changed the title [ci-auto] GenAIExample ChatQnA compose.yaml got changed. [ci-auto] remove "--enforce-eager" for better vLLM perf Dec 10, 2024
lianhao added a commit to lianhao/GenAIInfra that referenced this issue Dec 10, 2024
@lianhao
Adapt to latest vllm changes
cde4096 
- Remove --eager-enfoce on hpu to improve performance
- Refactor to the upstream docker entrypoint changes

Fixes issue opea-project#631.

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
@lianhao lianhao mentioned this issue Dec 10, 2024
Closed
1 task
lianhao added a commit to lianhao/GenAIInfra that referenced this issue Dec 10, 2024
@lianhao
Adapt to latest vllm changes
4aad447 
- Remove --eager-enforce on hpu to improve performance
- Refactor to the upstream docker entrypoint changes

Fixes issue opea-project#631.

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
lianhao added a commit to lianhao/GenAIInfra that referenced this issue Dec 10, 2024
@lianhao
Adapt to latest vllm changes
6bfddb6 
- Remove --eager-enforce on hpu to improve performance
- Refactor to the upstream docker entrypoint changes

Fixes issue opea-project#631.

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
eero-t pushed a commit to eero-t/GenAIInfra that referenced this issue Dec 10, 2024
@lianhao @eero-t
Adapt to latest vllm changes
576bb54 
- Remove --eager-enforce on hpu to improve performance
- Refactor to the upstream docker entrypoint changes

Fixes issue opea-project#631.

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
lianhao added a commit to lianhao/GenAIInfra that referenced this issue Dec 11, 2024
@lianhao
Adapt to latest vllm changes
193f120 
- Remove --eager-enforce on hpu to improve performance
- Refactor to the upstream docker entrypoint changes

Fixes issue opea-project#631.

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
eero-t pushed a commit to eero-t/GenAIInfra that referenced this issue Dec 11, 2024
@lianhao @eero-t
Adapt to latest vllm changes
f38b1b0 
- Remove --eager-enforce on hpu to improve performance
- Refactor to the upstream docker entrypoint changes

Fixes issue opea-project#631.

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
lianhao added a commit to lianhao/GenAIInfra that referenced this issue Dec 17, 2024
@lianhao
Adapt to latest vllm changes
616ca05 
- Remove --eager-enforce on hpu to improve performance
- Refactor to the upstream docker entrypoint changes

Fixes issue opea-project#631.

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
eero-t pushed a commit to eero-t/GenAIInfra that referenced this issue Dec 17, 2024
@lianhao @eero-t
Adapt to latest vllm changes
75f8d2a 
- Remove --eager-enforce on hpu to improve performance
- Refactor to the upstream docker entrypoint changes

Fixes issue opea-project#631.

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
@lianhao lianhao mentioned this issue Dec 18, 2024
Merged
2 tasks
yongfengdu pushed a commit that referenced this issue Dec 18, 2024
@eero-t @lianhao
Add vLLM+HPA support to ChatQnA Helm chart (#610)
baed0b5 
* Add monitoring support for the vLLM component

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>

* Initial vLLM support for ChatQnA

For now vLLM replaces just TGI, but as it supports also embedding,
also TEI-embed/-rerank may be replaceable later on.

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>

* Fix HPA comments in tgi/tei/tererank values files

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>

* Add HPA scaling support for ChatQnA / vLLM

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>

* Adapt to latest vllm changes

- Remove --eager-enforce on hpu to improve performance
- Refactor to the upstream docker entrypoint changes

Fixes issue #631.

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>

* Clean up ChatQnA vLLM Gaudi parameters

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>

---------

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Co-authored-by: Lianhao Lu <lianhao.lu@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lianhao lianhao

Labels
helm
Projects
None yet
Milestone
No milestone
2 participants
@daisy-ycguo @lianhao