[Misc]: Implement CPU/GPU swapping in BlockManagerV2

[cadedaniel]

Recently, we refactored the block manager subsystem to improve testability by separating concerns of each layer. See #3492 for more information.

The V2 implementation does not have support for CPU-GPU swapping. It can be added in the CpuGpuBlockAllocator. My first take on the design is that it should simply keep track of the requested swap requests and have the scheduler get_and_clear them after each scheduling step.

[Kaiyang-Chen]

I would like to have a try on this!

[cadedaniel]

I would like to have a try on this!

That would be great! Let me know if I can answer any questions

[foreverpiano]

@cadedaniel Hi, does the block manager v2 support cpu-gpu offloading now?

[KuntaiDu]

I can also help! Let me take a look at the example code and BlockSpaceManager first.

[cadedaniel]

Closed by #3834