What are the fir and sde_vp kernels in ResnetBlock2D

hchings · December 26, 2022, 8:42pm

Hi there,

I saw there are different kernels used in the ResnetBlock2D.

huggingface/diffusers/blob/2dcf64b72a44dc748610c3eb51534b325df3bca9/src/diffusers/models/resnet.py#L431


      
          self.conv2 = torch.nn.Conv2d(out_channels, out_channels, kernel_size=3, stride=1, padding=1)
          
          
if non_linearity == "swish":
              self.nonlinearity = lambda x: F.silu(x)
          elif non_linearity == "mish":
              self.nonlinearity = Mish()
          elif non_linearity == "silu":
              self.nonlinearity = nn.SiLU()
          
          
self.upsample = self.downsample = None
          if self.up:
              if kernel == "fir":
                  fir_kernel = (1, 3, 3, 1)
                  self.upsample = lambda x: upsample_2d(x, kernel=fir_kernel)
              elif kernel == "sde_vp":
                  self.upsample = partial(F.interpolate, scale_factor=2.0, mode="nearest")
              else:
                  self.upsample = Upsample2D(in_channels, use_conv=False)
          elif self.down:
              if kernel == "fir":
                  fir_kernel = (1, 3, 3, 1)

Can anyone point me to the full names and papers/online materials to understand these kernels? Thanks!

Topic		Replies	Views
UNet1DModel not converging on single batch 🧨 Diffusers	3	593	June 19, 2023
Unet1dmodel fails 🧨 Diffusers	1	670	April 14, 2023
Mini Stable Diffusion. How-to guide? Beginners	0	1216	February 15, 2023
Training Diffuser Model on Colab GPU - 'nvidia-smi' Error & Feasibility Beginners	1	30	August 23, 2024
Meaning of vector fields in Flux and SD3 loss function 🧨 Diffusers	1	69	September 13, 2024

What are the fir and sde_vp kernels in ResnetBlock2D

Related topics