How can I use multi gpu on stable diffusion pipeline?

I need just inference.


As do iiiiii :100:

I tried batching by clubbing the prompts. The it/s reduced from 4 it/s to 1it/s for a batch of 4 prompts

I tried adding multiple T4 GPUs to the gcloud vm and tried running unsuccessfully. monitoring nvidia-smi shows the second GPU is not begin used at all

Same here. Did you find some insights on how to do it? I tried device_map=“auto” option, but still only one GPU is used.

I think the only option, at the moment, is to create multiple instances. So, if you want to run a batch, run one instance for each GPU that you have. Set each instance to each individual GPU and increment the seed by 1 per batch, and by 4 (if using 4 GPUs), so each one is processing a different output with the same settings.

Similar setup if you want to produce more passes, but the same seed. Change the per-instance value by one value, and each individual instances values increment by 4x that same initial offset value.

Natively, “we are not there yet”, but I am sure they are working on it. I know a few GUIs do this to create animations. Each “frame” goes to a different GPU. But, only ONE GPU can work on ONE output at a time. Creation, at the moment, is linear. Until they start making “larger creations” that can be made from “areas”, and the results merged back into one singular output. (Like how a 3D rendering program works, in tiny little boxes, to create one large image.)

If could be nice to run batch img_to_img on two GPUs. It could be possible by setting CUDA_VISIBLE_DEVICES to a number of the specific GPU before the launch of each WebUI instance. on GPU 0 and on GPU 1, for example.

Will have to try it out.