How to specify the version of attention for StableDiffusionXLPipeline?

🧨 Diffusers
1

Looking a the source code of Diffusers, there are 2 options of attention that we can select for inferencing:

  1. The default option is PyTorch implementation of attention which calls torch.nn.functional.scaled_dot_product_attention(). The corresponding attention processor class is here.

  2. Another option is the naive attention implemented in the diffuser source code. The corresponding attention processor class is here.

My question is how do we select the option of attention for inferencing StableDiffusionXLPipeline, if we initialize it with from_pretrained() like the below image? What parameter can we pass into from_pretrained() that specify the attention version? If there’s no such parameter, are there other ways to do it?

image
image958×545 47 KB

2

Following the source code, it appears that it is this option that specifies it. I’m not sure about this option as I’ve never used it…
Maybe you need to install xformers separately.

4

I see the option enable xformer’s memory efficient attention. However, is there an option that uses scaled_dot_product_attention of pytorch, or the naive attention?

Xformer is really good but I can’t use it though, because I am inferencing on AMD GPUs and it doesn’t support rocm.

5

I see that you are in an environment where xformers are not available. The following is the base class for Diffusers model relations, so it should be an option for almost any model, but I don’t see anyone using it, so I don’t know if it is available…