New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LoRA is not supported with quantized models yet #2821

Closed

fmmoret opened this issue Feb 8, 2024 · 1 comment

fmmoret commented Feb 8, 2024

vllm/vllm/config.py

Line 504 in 3711811

"LoRA is not supported with quantized models yet.")

Would like to be able to use with bitsandbytes 4/8 bit, awq, etc.

fmmoret mentioned this issue

Support loras on quantized models #2828

Closed

Collaborator

hmellor commented Apr 18, 2024

bnb has been requested in [Feature]: bitsandbytes support #4033
QLoRA has been requested in Add Support for QLORA/QA-QLORA weights which are not merged #3225

hmellor closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment