habana_main rebase #81

kzawora-intel · 2024-07-02T13:37:10Z

Retrying #71

…llm-project#4971) Co-authored-by: Jianan Gu <jianan.gu@intel.com>

Co-authored-by: Roger Wang <ywang@roblox.com>

Co-authored-by: Michael Goin <michael@neuralmagic.com> Co-authored-by: youkaichao <youkaichao@gmail.com> Co-authored-by: zifeitong <zifei.tong@parasail.io> Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>

…-project#5478)

)

… with `perf-benchmarks` label (vllm-project#5073) Co-authored-by: simon-mo <simon.mo@hey.com>

…#5516)

Co-authored-by: mgoin <michael@neuralmagic.com> Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>

…#5401)

…t#5546)

…roject#5512)

…project#5460) Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

…a/rebase_v3

…ct#6041) Co-authored-by: Simon Mo <simon.mo@hey.com>

…roupPool (vllm-project#6039)

…le (vllm-project#5965) Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com> Co-authored-by: Joshua Rosenkranz <jmrosenk@us.ibm.com>

…ult) (vllm-project#5602)

Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Roger Wang <ywang@roblox.com>

…ase_v3

bigPYJ1151 and others added 30 commits June 13, 2024 09:33

[Hardware][Intel] Optimize CPU backend and add more performance tips (v…

80aa7e9

…llm-project#4971) Co-authored-by: Jianan Gu <jianan.gu@intel.com>

[Docs] Add 4th meetup slides (vllm-project#5509)

a65634d

[Misc] Add vLLM version getter to utils (vllm-project#5098)

03dccc8

[CI/Build] Simplify OpenAI server setup in tests (vllm-project#5100)

3987347

[Doc] Update LLaVA docs (vllm-project#5437)

0ce7b95

Co-authored-by: Roger Wang <ywang@roblox.com>

[MISC] Remove FP8 warning (vllm-project#5472)

30299a4

Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>

Seperate dev requirements into lint and test (vllm-project#5474)

a8fda4f

Revert "[Core] Remove unnecessary copies in flash attn backend" (vllm…

6b0511a

…-project#5478)

[misc] fix format.sh (vllm-project#5511)

1696efe

[CI/Build] Disable test_fp8.py (vllm-project#5508)

33e3b37

[Kernel] Disable CUTLASS kernels for fp8 (vllm-project#5505)

e38042d

Add cuda_device_count_stateless (vllm-project#5473)

50eed24

[Hardware][Intel] Support CPU inference with AVX2 ISA (vllm-project#5452

cd9c0d6

)

[Misc] Fix arg names in quantizer script (vllm-project#5507)

55d6361

bump version to v0.5.0.post1 (vllm-project#5522)

0f0d8bc

[CI/Build][Misc] Add CI that benchmarks vllm performance on those PRs…

319ad7f

… with `perf-benchmarks` label (vllm-project#5073) Co-authored-by: simon-mo <simon.mo@hey.com>

[CI/Build] Disable LLaVA-NeXT CPU test (vllm-project#5529)

d47af2b

[Kernel] Fix CUTLASS 3.x custom broadcast load epilogue (vllm-project…

703475f

…#5516)

[Misc] Fix arg names (vllm-project#5524)

d74674b

[ Misc ] Rs/compressed tensors cleanup (vllm-project#5432)

1598568

Co-authored-by: mgoin <michael@neuralmagic.com> Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>

[Kernel] Suppress mma.sp warning on CUDA 12.5 and later (vllm-project…

348616a

…#5401)

[mis] fix flaky test of test_cuda_device_count_stateless (vllm-projec…

48f589e

…t#5546)

[Core] Remove duplicate processing in async engine (vllm-project#5525)

77490c6

[misc][distributed] fix benign error in is_in_the_same_node (vllm-p…

d1c3d7d

…roject#5512)

[Docs] Add ZhenFund as a Sponsor (vllm-project#5548)

cdab68d

[Doc] Update documentation on Tensorizer (vllm-project#5471)

6e2527a

[Bugfix] Enable loading FP8 checkpoints for gpt_bigcode models (vllm-…

e2afb03

…project#5460) Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

[Bugfix] Fix typo in Pallas backend (vllm-project#5558)

28c145e

[Core][Distributed] improve p2p cache generation (vllm-project#5528)

f5bb85b

kzawora-intel and others added 27 commits July 1, 2024 15:37

Merge remote-tracking branch 'origin/habana_main' into private/kzawor…

463a8e6

…a/rebase_v3

remove vllm/hpu/attn_bias.py

0141d57

Merge remote-tracking branch 'origin/habana_main' into private/kzawor…

52fa486

…a/rebase_v3

whitespace fix

a21fe62

revert accidental changes in rmsnorm

aaf5446

Fix hpugraph hashing

1ec95c4

add trim_attn_metadata comment

2394c41

fix prompt bucketing:

98fb698

[ CI ] Re-enable Large Model LM Eval (vllm-project#6031)

d76084c

[doc][misc] remove deprecated api server in doc (vllm-project#6037)

4050d64

[Misc] update benchmark backend for scalellm (vllm-project#6018)

bb60326

[doc][misc] further lower visibility of simple api server (vllm-proje…

8893130

…ct#6041) Co-authored-by: Simon Mo <simon.mo@hey.com>

[Bugfix] Use RayActorError for older versions of Ray in RayTokenizerG…

dec6fc6

…roupPool (vllm-project#6039)

[Bugfix] adding chunking mechanism to fused_moe to handle large inputs (

12a5995

vllm-project#6029)

add FAQ doc under 'serving' (vllm-project#5946)

83bdcb6

[Bugfix][Doc] Fix Doc Formatting (vllm-project#6048)

8e0817c

[Bugfix] Add explicit end_forward calls to flashinfer (vllm-project…

c4059ea

…#6044)

[BugFix] Ensure worker model loop is always stopped at the right time (…

c87ebc3

…vllm-project#5987)

[Frontend] Relax api url assertion for openai benchmarking (vllm-proj…

e373853

…ect#6046)

[Model] Changes to MLPSpeculator to support tie_weights and input_sca…

5460070

…le (vllm-project#5965) Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com> Co-authored-by: Joshua Rosenkranz <jmrosenk@us.ibm.com>

[Core] Optimize block_manager_v2 vs block_manager_v1 (to make V2 defa…

3476ed0

…ult) (vllm-project#5602)

[Frontend] Add template related params to request (vllm-project#5709)

2c37540

[VLM] Remove image_input_type from VLM config (vllm-project#5852)

98d6682

Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Roger Wang <ywang@roblox.com>

Merge remote-tracking branch 'upstream/main' into private/kzawora/reb…

c365082

…ase_v3

[Doc] Reinstate doc dependencies (vllm-project#6061)

31354e5

Merge remote-tracking branch 'upstream/main' into private/kzawora/reb…

aee6daf

…ase_v3

guard model loader wa for hpu

d99d986

kzawora-intel merged commit a797e6a into habana_main Jul 2, 2024

kzawora-intel added the habana Issues or PRs submitted by Habana Labs label Sep 20, 2024

kzawora-intel deleted the private/kzawora/rebase_v3 branch October 7, 2024 12:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

habana_main rebase #81

habana_main rebase #81

kzawora-intel commented Jul 2, 2024

habana_main rebase #81

habana_main rebase #81

Conversation

kzawora-intel commented Jul 2, 2024