Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Caikit prompts #193

Closed
wants to merge 137 commits into from
Closed

Caikit prompts #193

wants to merge 137 commits into from

Conversation

joerunde
Copy link

Test build for adapter changes

ywang96 and others added 30 commits September 26, 2024 07:45
@ywang96
[Misc] Update config loading for Qwen2-VL and remove Granite (vllm-pr…
4bb98f2 
…oject#8837)
@tlrmchlsmth
[Build/CI] Upgrade to gcc 10 in the base build Docker image (vllm-pro…
f70bcca 
…ject#8814)
@mgoin
[Docs] Add README to the build docker image (vllm-project#8825)
520db4d
@fyuan1316
[CI/Build] Fix missing ci dependencies (vllm-project#8834)
68988d4
@youkaichao
[misc][installation] build from source without compilation (vllm-proj…
70de39f 
…ect#8818)
@khluu
[ci] Soft fail Entrypoints, Samplers, LoRA, Decoder-only VLM (vllm-pr…
d9cfbc8 
…oject#8872)

Signed-off-by: kevin <kevin@anyscale.com>
@Pernekhan
[Bugfix] Include encoder prompts len to non-stream api usage response (…
93d364d 
…vllm-project#8861)
@mgoin
[Misc] Change dummy profiling and BOS fallback warns to log once (vll…
b28d210 
…m-project#8820)
@tlrmchlsmth
[Bugfix] Fix print_warning_once's line info (vllm-project#8867)
e2f6f26
@chiragjn
fix validation: Only set tool_choice auto if at least one tool is p…
ee2da3e 
…rovided (vllm-project#8568)
@tlrmchlsmth
[Bugfix] Fixup advance_step.cu warning (vllm-project#8815)
71d21c7
@njhill
[BugFix] Fix test breakages from transformers 4.45 upgrade (vllm-proj…
4b377d6 
…ect#8829)
@DarkLight1337
[Installation] Allow lower versions of FastAPI to maintain Ray 2.9 co…
1b49148 
…mpatibility (vllm-project#8764)
@maxdebayser
[Feature] Add support for Llama 3.1 and 3.2 tool use (vllm-project#8343)
344cd2b 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
@DarkLight1337
[Core] renamePromptInputs and inputs (vllm-project#8876)
3b00b9c
@youkaichao
[misc] fix collect env (vllm-project#8894)
dc4e3df
@panpan0000
[MISC] Fix invalid escape sequence '\' (vllm-project#8830)
0e08875 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
@Isotr0py
[Bugfix][VLM] Fix Fuyu batching inference with max_num_seqs>1 (vllm…
6d792d2 
…-project#8892)
@bvrockwell
[TPU] Update pallas.py to support trillium (vllm-project#8871)
8df2dc3
@youkaichao
[torch.compile] use empty tensor instead of None for profiling (vllm-…
a9b15c6 
…project#8875)
@ProExpertProg
[Kernel] AQ AZP 4/4: Integrate asymmetric quantization to linear meth…
172d1cd 
…od (vllm-project#7271)
@LucasWilkinson @mgoin
[Bugfix] fix for deepseek w4a16 (vllm-project#8906)
c5d5535 
Co-authored-by: mgoin <michael@neuralmagic.com>
@varun-sundar-rabindranath
[Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (
c2ec430 
vllm-project#8378)

Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
@youkaichao
[misc][distributed] add VLLM_SKIP_P2P_CHECK flag (vllm-project#8911)
18e60d7
@schoennenbeck
[Core] Priority-based scheduling in async engine (vllm-project#8850)
bd429f2
@youkaichao
[misc] fix wheel name (vllm-project#8919)
d86f6b2
@tylertitsworth @youkaichao
[Bugfix][Intel] Fix XPU Dockerfile Build (vllm-project#7824)
260024a 
Signed-off-by: tylertitsworth <tyler.titsworth@intel.com>
Co-authored-by: youkaichao <youkaichao@126.com>
@DarkLight1337
[Misc] Remove vLLM patch of BaichuanTokenizer (vllm-project#8921)
b0298aa
@tastelikefeet
[Bugfix] Fix code for downloading models from modelscope (vllm-projec…
39d3f8d 
…t#8443)
@varun-sundar-rabindranath
[Bugfix] Fix PP for Multi-Step (vllm-project#8887)
19d02ff
russellb and others added 20 commits October 7, 2024 21:18
@russellb
[CI/Build] Add linting for github actions workflows (vllm-project#7876)
e0dbdb0 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@KuntaiDu
[Doc] Include performance benchmark in README (vllm-project#9135)
c0d9a98
@youkaichao
[misc] fix comment and variable name (vllm-project#9139)
fa45513
@simon-mo
Add Slack to README (vllm-project#9137)
8eeb857
@youkaichao
[misc] update utils to support comparing multiple settings (vllm-proj…
04c12f8 
…ect#9140)
@jikunshang
[Intel GPU] Fix xpu decode input (vllm-project#9145)
80b57f0
@youkaichao
[misc] improve ux on readme (vllm-project#9147)
e1faa2a
@fialhocoelho
Merge branch 'main' of https://github.com/vllm-project/vllm into ibm-…
e7db59c 
…20241008

Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 5733
573d075 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 6357
61f6474 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 9034
6d27ed2 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 9049
22a7fe8 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 9027
bf8438e 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
install adapter from @main to use adapter/#137
6b500af 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Hack up de build to use as base-image 🚀
baeec70 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
🐛 Reverted logic to fix build; potential GGUF-related issues.
1297cc8 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Revert "Hack up de build to use as base-image 🚀"
bc59bd3 
This reverts commit baeec70.

Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@maxdebayser
enable head size 32
5629875 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
@maxdebayser
two-stage build
c2090bf 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
@joerunde
⚗️ test build for caikit prompts
37128cb 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 11, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: joerunde
Once this PR has been reviewed and has the lgtm label, please assign danielezonca for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 11, 2024

@joerunde: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/smoke-test 37128cb link true /test smoke-test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-robot
Copy link

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@joerunde
Copy link
Author

was just for builds

@joerunde joerunde closed this Oct 17, 2024
prarit pushed a commit to prarit/vllm that referenced this pull request Oct 21, 2024
@kkHuang-amd
[Grok-1] 1. upload moe configuration file for moe kernel optimization… (
40581f4 
opendatahub-io#193)

* [Grok-1] 1. upload moe configuration file for moe kernel optimization 2. support "--num-scheduler-steps" in benchmark_latency.py

* [Grok-1] 1. upload moe configuration file for moe kernel optimization 2. add copy of benchmark_latency.py to support "--num-scheduler-steps"

* [Grok-1] add option num-scheduler-steps in benchmark_latency.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tarukumar tarukumar Awaiting requested review from tarukumar

@Xaenalt Xaenalt Awaiting requested review from Xaenalt

Assignees
No one assigned
Labels
do-not-merge/hold needs-rebase
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.