Fix gemv_fast model loading

[chu-tianxiang]

GEMV fast shares many similarities with marlin both in algorithm and packing. Additional measures are required for managing the tiling/interleaving process in order to properly load the weights. Currently I following the practice in Marlin. It may worth some refactoring in the future, for example, separate the pack_factor into output_pack_factor and input_pack_factor.

The code works for llama now, however more shape checks are needed especially because of the calculate_zeros_width padding.

[garycaokai]

Great job, have you test the performance of awq_fast, I have tried the mistral-instruct-v0.2-gemvfast-awq model, the performance is not so fast

[chu-tianxiang]

Great job, have you test the performance of awq_fast, I have tried the mistral-instruct-v0.2-gemvfast-awq model, the performance is not so fast

I only ran a few tests comparing vllm.LLM.generate (old gemm and new gemv fast) and AutoAWQForCausalLM.generate. GEMV fast is faster than old GEMM across all batch sizes. vllm is comparable or better than AutoAWQForCausalLM at small batch size while slightly worse at large batch size (64).