image transforms

[ethansmith2000]

making a fork for my own usage, but thought i'd give a pr as well. thank you for your work!

[brian6091]

Great, looks a lot cleaner! One issue though (which applies in general, not just to your PR) is that if both resize and center_crop are false, you will potentially give the VAE images at different resolutions. Perhaps useful to include a check to force a resize at the end of the pipeline before converting to a tensor? This is useful if you feed the training script images with varying sizes.

I've done this here:
https://github.com/brian6091/Dreambooth/blob/mix/src/datasets.py#L112-132

[ethansmith2000]

good point! also noticed there isn't an option for caching latents, I'd be happy to make a pr for that as well if that's something that would be useful

[brian6091]

There is an issue for caching latents: #62

One point is that if you are relying on transformations to augment your data, like color jitter, caching latents won't help you since you need to run the transformations on each batch.

Maybe you could chime in on the discussion to see what others think. My general feeling is to keep the training scripts in the LoRA repo as simple as possible so that others can easily see what they need to do to adapt their own scripts. But if caching latents saves enough memory to let more people train with lower GPU requirements, then it may be worth it.

[ethansmith2000]

ah didn't think of that. good reason to have it optional.
there probably isn't a reliable way to jitter colors of the latents, but also maybe could do the linear decode trick, apply, and turn it back

[brian6091]

Or you could augment in latent space, but I'm not sure anyone knows what that means!

[cloneofsimo]

Thanks!