I am currently following the using diffusers section on the documentation page and have come to a point where I can swap out pipeline elements from valid diffusers libraries hosted at hugging face. I can either load the model from the diffusers pipeline and get the components I want, or download and replace the relevant folder manually from the repository. Expanding this, I wanted to try to use community-made stable diffusion models on websites like Civitai, as well as custom community components.
As an example, I would like to use this model checkpoint that is available in a safetensor format:
With the following vae(s):
I have found that cloning the
stable diffusion-v1-5 repository and replacing the
.ckpts with the revAnimated '.safetensor` works to run the model.
However, the community vae(s) come either in:
which is not compatible with the
diffusers.models.AutoencoderKL as it expects a
diffusion_pytorch_model.bin file. I would like to try to convert these files into a format usable by the diffusers library.
For the conversion I have tried following the instructions on this issue:
This leads me to the following spaces:
The first page gets stuck in a preparing space loop:
The other page has a runtime error on load:
Runtime error Traceback (most recent call last): File "app.py", line 3, in <module> from convert import convert File "/home/user/app/convert.py", line 74, in <module> def convert(token: str, model_id: str, filename: str, model_type: str, sample_size: int = 512, scheduler_type: str = "pndm", extract_ema: bool = True, progress=gr.Progress()): AttributeError: module 'gradio' has no attribute 'Progress'
- Is there a way I can do the conversion locally?
- If yes, does the resulting conversion give me a stable diffusion model or just the vae components?
- Is there a way to use these components with the diffuser library without the need for conversions?
I am aware that these work out of the box with a custom web UI like Automatic 1111, however, my objective is to better learn the diffusers library.