[Inference/Kernel] refactor kvcache manager and rotary_embedding and kvcache_memcpy oper…#5663

Merged

SunflowerAries merged 3 commits intohpcaitech:feature/colossal-inferfrom SunflowerAries:refactor-key-cache-layout

Apr 30, 2024

Commits on Apr 30, 2024