Skip to content

Commit

Permalink
[Docs] Updated docs for compatibility matrix
Browse files Browse the repository at this point in the history
Signed-off-by: Wallas Santos <wallashss@ibm.com>
  • Loading branch information
wallashss committed Sep 5, 2024
1 parent f2bec23 commit 282dbff
Showing 1 changed file with 23 additions and 26 deletions.
49 changes: 23 additions & 26 deletions docs/source/serving/compatibility_matrix.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ The table below shows mutually exclusive features along with support for some de

.. list-table::
:header-rows: 1
:widths: 20 8 8 8 8 8 8 8 8
:widths: 20 8 8 8 8 8 8 8 8 8

* - Feature
- Chunked Prefill
Expand All @@ -17,7 +17,8 @@ The table below shows mutually exclusive features along with support for some de
- Speculative decoding
- CUDA Graphs
- Encoder/Decoder
- Logprobs*
- Logprobs
- Prompt Logprobs
* - APC
- ✅
-
Expand All @@ -29,7 +30,7 @@ The table below shows mutually exclusive features along with support for some de
-
-
* - LoRa
- ❌ [[1C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/config.py#L1381)]
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/config.py#L1381>`__
- ✅
-
-
Expand All @@ -49,9 +50,9 @@ The table below shows mutually exclusive features along with support for some de
-
-
* - Speculative decoding
- ❌ [[2C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/config.py#L1024)] [[3T](https://github.com/vllm-project/vllm/issues/5016)]
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/config.py#L1024>`__ `[T] <https://github.com/vllm-project/vllm/issues/5016>`__
- ✅
- ❌ [[4C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/spec_decode/spec_decode_worker.py#L85-L86)]
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/spec_decode/spec_decode_worker.py#L85-L86>`__
- ✅
-
-
Expand All @@ -69,12 +70,12 @@ The table below shows mutually exclusive features along with support for some de
-
-
* - Encoder/Decoder
- ❌ [[5C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L24)]
- ❌ [[6C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L16)][[18T](https://github.com/vllm-project/vllm/issues/7366)]
- ❌ [[7C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L35C1-L36C1)]
- ❌ [[8C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L55)]
- ❌ [[9C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L47)][[18T](https://github.com/vllm-project/vllm/issues/7366)]
- ❌ [[10C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L51)][[19T](https://github.com/vllm-project/vllm/issues/7447)]
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L24>`__
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L16>`__ `[T] <https://github.com/vllm-project/vllm/issues/7366>`__
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L35C1-L36C1>`__
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L55>`__
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L47>`__ `[T] <https://github.com/vllm-project/vllm/issues/7366>`__
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/worker/utils.py#L51>`__ `[T] <https://github.com/vllm-project/vllm/issues/7447>`__
-
-
-
Expand All @@ -83,7 +84,7 @@ The table below shows mutually exclusive features along with support for some de
- ✅
- ✅
- ✅
- ❌ [[11C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/engine/output_processor/multi_step.py#L52)]
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/engine/output_processor/multi_step.py#L52>`__
- ✅
- ✅
-
Expand All @@ -93,7 +94,7 @@ The table below shows mutually exclusive features along with support for some de
- ✅
- ✅
- ✅
- ❌ [[11C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/engine/output_processor/multi_step.py#L52)]
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/engine/output_processor/multi_step.py#L52>`__
- ✅
- ✅
- ✅
Expand All @@ -109,13 +110,13 @@ The table below shows mutually exclusive features along with support for some de
- ✅
- ✅
* - CPU
- ❌ [[12C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/executor/cpu_executor.py#L328)]
- ❌ [[13C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/executor/cpu_executor.py#L337)]
- ❌ [[14C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/executor/cpu_executor.py#L29)]
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/executor/cpu_executor.py#L328>`__
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/executor/cpu_executor.py#L337>`__
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/executor/cpu_executor.py#L29>`__ `[T] <https://github.com/vllm-project/vllm/pull/4830>`__
- ?
- ✅
- ❌ [[15C](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/executor/cpu_executor.py#L318)]
- ❌ [[16](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/tests/models/test_bart.py#L8)]
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/vllm/executor/cpu_executor.py#L318>`__
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/tests/models/test_bart.py#L8>`__
- ✅
- ✅
* - AMD
Expand All @@ -125,15 +126,11 @@ The table below shows mutually exclusive features along with support for some de
- ✅
- ✅
- ✅
- ❌ [[17](https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/tests/kernels/test_encoder_decoder_attn.py#L753)]
- ❌ `[C] <https://github.com/vllm-project/vllm/blob/757ac70a64b5a643b68281c0b65f72f847cedbd6/tests/kernels/test_encoder_decoder_attn.py#L753>`__
- ✅
- ✅

Note:

- Logprobs include the support for both output logbrobs and prompt logprobs.

Related Issues:

- Encoder/Decoder feature compatibility https://github.com/vllm-project/vllm/issues/7366
- Speculative decoding with chunked prefill https://github.com/vllm-project/vllm/issues/5016
[C] stands for code checks, that is, there is a checking on running that verify if the combinations is valid and raises and error or log a warning disabling the feature.
[T] stands for tracking issues or pull requests on vLLM Repo

0 comments on commit 282dbff

Please sign in to comment.