Hi All, Does anybody have any guidance as to how/where to add further conditioning info to the HF stable diffusion training/inference pipelines? Everything I’ve read about stable diffusion seems to suggest that multiple different types of conditioning should be possible, but I’m not sure how to int…

Hi @jbmaxwell ! That’s an excellent question. The easiest way, I think, would be to leverage the UNet2DConditionModel and indicate here that you’ll be using custom class embeddings. Similar to what you suspected, these embeddings are simply added to the timestep embeddings . If you use the "timestep" …

Add additional conditioning info

🧨 Diffusers

jbmaxwell January 24, 2023, 4:22pm 3

Excellent, thanks so much @pcuenq!

Topic		Replies	Views
Class-conditional image generation 🧨 Diffusers	1	1144	October 12, 2023
Best way to add conditions to diffusion model 🧨 Diffusers	0	1547	June 3, 2023
Diffusion conditional model from diffusion-model-class Beginners	0	1437	February 25, 2023
Replace Stable Diffusion class-conditional text with rows of attributes 🧨 Diffusers	0	446	January 27, 2024
Replace text encoder with a different encoder in Stable Diffusion 🧨 Diffusers	0	1452	February 9, 2024

Add additional conditioning info

Related topics