How to use single-file diffuser checkpoints

Beginners
1

I can’t figure out how to load a diffusion model from single files, like flux1-dev.safetensors for FLUX.1 (black-forest-labs/FLUX.1-dev at main)

It works fine when using FluxPipeline.from_pretrained() which uses the multi-file format, but when I try FluxPipeline.from_single_file() on the above file I get:

diffusers.loaders.single_file_utils.SingleFileComponentError: Failed to load CLIPTextModel. Weights for this component appear to be missing in the checkpoint.
Please load the component before passing it in as an argument to `from_single_file`.

text_encoder = CLIPTextModel.from_pretrained('...')
pipe = FluxPipeline.from_single_file(<checkpoint path>, text_encoder=text_encoder)

So I was wondering, if the file only contains the transformer, which might be converted and inserted into the multi-file tree. Looking inside, though, I found that the number of tensors per layer is different and the dimensions are not quite the same, either.

There must be some way to use that file through diffusers, right?

1 Like
2

Hi,

The single file is meant to be used with ComfyUI.

1 Like
3

I see. Is there are reasonably easy way to still use it with diffusers? Some finetunes only provide it in that format.

1 Like
4

Files for ComfyUI can often be loaded using the following method. Please specify an appropriate repo for the missing parts other than DiT.

from diffusers import FluxPipeline, FluxTransformer2DModel

HF_TOKEN = "hf_***"
flux_repo = "multimodalart/FLUX.1-dev2pro-full"
ckpt_path = "https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors"
transformer = FluxTransformer2DModel.from_single_file(ckpt_path, subfolder="transformer", torch_dtype=torch.bfloat16, token=HF_TOKEN)
pipe = FluxPipeline.from_pretrained(flux_repo, transformer=transformer, torch_dtype=torch.bfloat16, token=HF_TOKEN)
5

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.