Add bitsandbytes fp4 support

[thesues]

bitsandbytes could support read bnb fp4 model such as PrunaAI/Einstein-v6.1-Llama3-8B-bnb-4bit-smashed.

I can not find any model which is tiny and fast. So I did not add new unit test.

this simple test works:

from vllm import LLM
model_id="PrunaAI/Einstein-v6.1-Llama3-8B-bnb-4bit-smashed"

llm = LLM(model=model_id, trust_remote_code=True, enforce_eager=True, quantization="bitsandbytes", \
          load_format="bitsandbytes",  max_model_len=2048)
outputs = llm.generate("What is the color of prunes?")
print(outputs[0].outputs[0].text)

output

INFO 08-09 00:51:04 model_runner.py:733] Loading model weights took 5.6168 GB
INFO 08-09 00:51:06 gpu_executor.py:102] # GPU blocks: 6598, # CPU blocks: 2048
Processed prompts: 100%|██████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.60it/s, est. speed input: 12.82 toks/s, output: 25.64 toks/s]
 They are actually dark purple to dark brownish black, with reddish hues if

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which consists a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of default ones by unblocking the steps in your fast-check build on Buildkite UI.

Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge).

To run full CI, you can do one of these:

• Comment /ready on the PR
• Add ready label to the PR
• Enable auto-merge.

🚀

[mgoin]

Are there other possible types that bnb uses other than fp4 and nf4? What happens with an 8bit or int4 model?

[thesues]

close this pr, duplicated in #7445 which supports fp4