Running into out of memory issues

🧨 Diffusers
1

Hi,

  • I’m currently trying to train huggingface Diffusers for 2D image generation task with images as input.
  • Training on AWS G5 instances i.e., A10G GPU’s with 24GB GPU memory.
  • I’m running into out of memory issues when I go beyond image size of 256x256 and batch size of 8.
  • Results with Image size = 256 and batch size = 8 is unacceptable
  • I did use gradient accumulation and mixed precision training.
  • Using 1 attention block only.

Trying to understand is it a genuine memory issue or can it be solved by some other approaches?
Is diffusion models so heavy that even a 24GB memory GPU is insufficient?
What’s the typical memory requirement for running image to image diffusion models, for generating images resolution higher than 512x512

Thanks and regards
KVS Moudgalya

2

Hey, sorry that you’re running into the issues.

The first thing I would try doing is to use xFormers: diffusers/examples/text_to_image at main · huggingface/diffusers · GitHub

3

Thanks @sayakpaul for your response.

I did enable xFormers in the huggingface diffusers, there is slight a bit of improvement in computational speed, but couldn’t not much in terms of memory, can’t increase image size above 256 and batch size above 8, getting out of memory issues if I go beyond these dimensions.