DiffusionPipeline.from_pretrained SDXL model: Input type and bias type should be the same

🧨 Diffusers
1

I am trying to run DiffusionPipeline on custom SDXL model and am having trouble with input and bias type. Anyone else have this issue/find a way around?

Code:

from diffusers import DiffusionPipeline
import torch
model = "stabilityai/stable-diffusion-xl-base-1.0"
pipe = DiffusionPipeline.from_pretrained(model, torch_dtype = torch.float16,
)
pipe.to("cuda")
pipe.load_lora_weights("/mys3bucket/model/", weight_name = "pytorch_lora_weights.safetensors"
)

prompt = ["photo of (zwx child) in outer space, nasa suit, space station, RAW candid cinema, 16mm, color graded portra 400 film, remarkable color, ultra realistic, textured skin, remarkable detailed pupils, shot with cinematic camera"]
negative_prompt = "lowres"
generator = torch.Generator("cuda")

for seed in range(1):

image = pipe(prompt = prompt[seed], negative_prompt=negative_prompt, generator=generator, num_inference_steps=25)
image = image.images[0]
image.save(f"/mys3bucket/output_images/{seed}.png")

Error:

Traceback (most recent call last):
  File "/home/ubuntu/sdxl2.py", line 25, in <module>
    image = pipe(prompt = prompt[seed], negative_prompt=negative_prompt, generator=generator, num_inference_steps=25)
  File "/usr/local/lib/python3.10/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 847, in __call__
    image = self.vae.decode(latents / self.vae.config.scaling_factor, return_dict=False)[0]
  File "/usr/local/lib/python3.10/dist-packages/diffusers/utils/accelerate_utils.py", line 46, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/diffusers/models/autoencoder_kl.py", line 270, in decode
    decoded = self._decode(z).sample
  File "/usr/local/lib/python3.10/dist-packages/diffusers/models/autoencoder_kl.py", line 256, in _decode
    z = self.post_quant_conv(z)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/conv.py", line 463, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/conv.py", line 459, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (c10::Half) and bias type (float) should be the same

More details here:

1 Like
2

Solved by updating transformers/xformers/torchvision/accelerators