[WIP] Hybrid allocator for full attention & sliding window attention interleaved models

[heheda12345]

This pr is built on top of #12086, will rebase and resolve merge conflicts with main branch after that pr got merged. The diff of this PR and #12086 is here: heheda12345/vllm@grouped_block_table...heheda12345:vllm:hybrid_allocator

This pr is working on hybrid memory allocator (#11382), and does the following things:

1. Extend KVCacheManger to support multiple KV cache groups
2. Introduce SpecializedManager, an abstraction for expressing the allocation & free & prefix caching logic of KV cache for different attention variants
3. Extend KVCacheConfig and GPUModelRunner.initialize_kv_cache to allow multiple layers sharing the same KV cache memory pool.

Benchmark results:
The following benchmarks are performed on H100, with Gemma2 a hybrid model combines sliding window attention layers and standard attention layers, and llama, a model with only standard attention layers. Hybrid allocator can accelerate hybrid model & only introduce very little overhead on standard full attention models.

This PR (68fe2db):

VLLM_USE_V1=1 python3 benchmark_throughput.py --model google/gemma-2-27b-it --input-len 6144 --output-len 1024 --num-prompts 50
Throughput: Throughput: 0.17 requests/s, 1250.04 total tokens/s, 178.58 output tokens/s
VLLM_USE_V1=1 python3 benchmark_throughput.py --model meta-llama/Llama-3.1-8B-Instruct --input-len 6144 --output-len 1024 --num-prompts 50
Throughput: 1.47 requests/s, 10541.44 total tokens/s, 1505.92 output tokens/s

Main branch (df450aa)

VLLM_USE_V1=1 python3 benchmark_throughput.py --model google/gemma-2-27b-it --input-len 6144 --output-len 1024 --num-prompts 50
Throughput: 0.15 requests/s, 1073.40 total tokens/s, 153.34 output tokens/s
VLLM_USE_V1=1 python3 benchmark_throughput.py --model meta-llama/Llama-3.1-8B-Instruct --input-len 6144 --output-len 1024 --num-prompts 50
Throughput: 1.48 requests/s, 10629.64 total tokens/s, 1518.52 output tokens/s

CC @comaniac @WoosukKwon

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @heheda12345.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork