How to convert ckpt to diffusers format

🧨 Diffusers
1

The community is heavily using both the .ckpt and the diffusers format. We are working on having better support for interoperability between the formats, but the recommended approach is always to just upload checkpoints in both formats.

Let’s dive a bit into the best approach to convert .ckpt into diffusers format.

Option 1.)

The easiest and most convenient approach is to just use a space to convert the checkpoint.
You can just try it here: SD To Diffusers - a Hugging Face Space by diffusers

As soon as the model is a bit more customized this might however not work. In this case the space tends to open an empty PR or just fails. This means that the conversion didn’t work.

Option 2.)

Option two is the more error-proof option and should always work.

  1. Git clone the repo that has a .ckpt and where you want to convert the checkpoint from. E.g.:
git lfs install
git clone https://huggingface.co/CiaraRowles/TemporalNet
  1. Open a pull request and follow the instructions
cd TemporalNet && git fetch origin refs/pr/13:pr/13

and

git checkout pr/14
  1. Now convert the checkpoint. For this you need to have a local clone diffusers . Then we recommend to always use the https://github.com/huggingface/diffusers/blob/main/scripts/convert_original_stable_diffusion_to_diffusers.py script.

You need to make sure to configure your input arguments correctly. We recommend to always pass the following parameters:

--original_config_file
--checkpoint_path
--dump_path

Let’s explain them a bit more in-detail:

--dump_path is just the path where the diffusers checkpoint will be save
--checkpoint_path is the path to the .ckpt file to be converted. If multiple .ckpt files are present, we recommend to always use the largest one
--original_config_file - the original config file. This should be a file that ends with a yaml suffix. In case no file is present you should try to find the corresponding file in the github repo the .ckpt model originated from. Some examples:

We recommend to then convert the checkpoint directly into the cloned local repo. For our example above we used the following command. We took the cldm_v15.yaml config from ControlNet/models at main · lllyasviel/ControlNet · GitHub since we know that the model is a sd15 model and a controlnet model and because no config was present in the original repo

python ../diffusers/scripts/convert_original_stable_diffusion_to_diffusers.py --checkpoint_path temporalnetv3.ckpt --original_config_file cldm_v15.yaml --dump_path ./ --controlnet
  1. Now we recommend that you test the converted checkpoint and then upload your changes to the PR:
git push origin pr/13:refs/pr/13

Done!

The resulting PR is here: CiaraRowles/TemporalNet · Add diffusers weights

4 Likes
2

The link in Option1 gives a Runtime error

## Memory limit exceeded (16Gi)