Add support to sequential cpu offload 8GB VRAM maybe less

[rodjjo]

What

Use even less memory to run it.

Why

To help people with 8GB VRAM GPU

Description

Add support to sequential CPU offloading.

• Take longer, however runs on with very low VRAM and way faster than only using CPU.
• Keeps the previous logic of offloading.

For the reviewer

Generate a text to video and a image to video to validate (I moved the logic from a personal project to support this one here).

How to use:

model = PyramidDiTForVideoGeneration(
    'PATH',                                         # The downloaded checkpoint dir
    model_dtype,
    model_variant='diffusion_transformer_768p',     # 'diffusion_transformer_384p'
)


model.vae.enable_tiling()
model.enable_sequential_cpu_offload()

[Ednaordinary]

I couldn't get this to run, it looks like the cpu_offload function didn't get copied over imported

Oops: just need to change the accelerate import to include cpu_offload

[feifeiobama]

I couldn't get this to run, it looks like the cpu_offload function didn't get copied over imported

Oops: just need to change the accelerate import to include cpu_offload

This is indeed a problem. @rodjjo could you submit another commit to include from accelerate import Accelerator, cpu_offload. Then, I will immediately merge this pull request and update the README.md.

[Ednaordinary]

With a really short video, I was also able to run as low as 2GB with #76 merged and tile_sample_min_size lowered to 64 (48, 32, 16 progressively became slower and vram usage didn't go down)

With this, vae decode is now the limiting factor. The exact vae decode usage seems to fluctuate depending on temp:

16 (5sec): 4.808GiB
10: 3.242GiB
5: 1.889GiB
3:1.780GiB

(anything in between should work, these are just the ones I chose to sample)

[feifeiobama]

With a really short video, I was also able to run as low as 2GB with #76 merged and tile_sample_min_size lowered to 64 (48, 32, 16 progressively became slower and vram usage didn't go down)

With this, vae decode is now the limiting factor. The exact vae decode usage seems to fluctuate depending on temp:

16 (5sec): 4.808GiB 10: 3.242GiB 5: 1.889GiB 3:1.780GiB

(anything in between should work, these are just the ones I chose to sample)

This looks great. Thank you @rodjjo!

[rodjjo]

Sorry for missing the include :D