This commit enables MoE on demand. On our GPUs, this cuts the
memory requirements by about half. Inference is still much slower.

You can instantiate MoE on demand on this branch using Mixtral
with command:

'''
LLM(model="/home/user/Workspace_4tb/grant/Mixtral-8x7B-v0.1",  max_model_len=2048, gpu_memory_utilization=0.98, enforce_eager=True, num_gpu_blocks_override=1000)
'''

Fixes MoE On Demand

This commit enables MoE on demand for DeepSeek V2

You can instantiate MoE on demand on this branch using DeepSeek Lite
with command:

'''
LLM(model="deepseek-ai/DeepSeek-V2-Lite",  max_model_len=2048, gpu_memory_utilization=0.98, enforce_eager=True, num_gpu_blocks_override=1000)
'''

Fixes MoE On Demand

Prefill stage is not yet complete