Add additional conditioning info

maliozer · February 16, 2024, 12:22am

how about added_cond_kwargs , can we pass the embeddings we have to make another condition here what do you think ?

huggingface/diffusers/blob/main/src/diffusers/models/unets/unet_2d_condition.py#L852


      
          
          def forward(
              self,
              sample: torch.FloatTensor,
              timestep: Union[torch.Tensor, float, int],
              encoder_hidden_states: torch.Tensor,
              class_labels: Optional[torch.Tensor] = None,
              timestep_cond: Optional[torch.Tensor] = None,
              attention_mask: Optional[torch.Tensor] = None,
              cross_attention_kwargs: Optional[Dict[str, Any]] = None,
              added_cond_kwargs: Optional[Dict[str, torch.Tensor]] = None,
              down_block_additional_residuals: Optional[Tuple[torch.Tensor]] = None,
              mid_block_additional_residual: Optional[torch.Tensor] = None,
              down_intrablock_additional_residuals: Optional[Tuple[torch.Tensor]] = None,
              encoder_attention_mask: Optional[torch.Tensor] = None,
              return_dict: bool = True,
          ) -> Union[UNet2DConditionOutput, Tuple]:
              r"""
              The [`UNet2DConditionModel`] forward method.
          
              Args:

@pcuenq

Topic		Replies	Views
Class-conditional image generation 🧨 Diffusers	1	1168	October 12, 2023
Best way to add conditions to diffusion model 🧨 Diffusers	0	1577	June 3, 2023
Diffusion conditional model from diffusion-model-class Beginners	0	1456	February 25, 2023
Replace Stable Diffusion class-conditional text with rows of attributes 🧨 Diffusers	0	457	January 27, 2024
Replace text encoder with a different encoder in Stable Diffusion 🧨 Diffusers	0	1549	February 9, 2024

Add additional conditioning info

Related topics