- I’m currently trying to train huggingface Diffusers for 2D image generation task with images as input.
- Training on AWS G5 instances i.e., A10G GPU’s with 24GB GPU memory.
- I’m running into out of memory issues when I go beyond image size of 256x256 and batch size of 8.
- Results with Image size = 256 and batch size = 8 is unacceptable
- I did use gradient accumulation and mixed precision training.
- Using 1 attention block only.
Trying to understand is it a genuine memory issue or can it be solved by some other approaches?
Is diffusion models so heavy that even a 24GB memory GPU is insufficient?
What’s the typical memory requirement for running image to image diffusion models, for generating images resolution higher than 512x512
Thanks and regards