What are the fir and sde_vp kernels in ResnetBlock2D

hchings · December 26, 2022, 8:42pm

Hi there,

I saw there are different kernels used in the ResnetBlock2D.

huggingface/diffusers/blob/2dcf64b72a44dc748610c3eb51534b325df3bca9/src/diffusers/models/resnet.py#L431


      
          self.conv2 = torch.nn.Conv2d(out_channels, out_channels, kernel_size=3, stride=1, padding=1)
          
          
if non_linearity == "swish":
              self.nonlinearity = lambda x: F.silu(x)
          elif non_linearity == "mish":
              self.nonlinearity = Mish()
          elif non_linearity == "silu":
              self.nonlinearity = nn.SiLU()
          
          
self.upsample = self.downsample = None
          if self.up:
              if kernel == "fir":
                  fir_kernel = (1, 3, 3, 1)
                  self.upsample = lambda x: upsample_2d(x, kernel=fir_kernel)
              elif kernel == "sde_vp":
                  self.upsample = partial(F.interpolate, scale_factor=2.0, mode="nearest")
              else:
                  self.upsample = Upsample2D(in_channels, use_conv=False)
          elif self.down:
              if kernel == "fir":
                  fir_kernel = (1, 3, 3, 1)

Can anyone point me to the full names and papers/online materials to understand these kernels? Thanks!

Topic		Replies	Views
UNet1DModel not converging on single batch 🧨 Diffusers	3	431	June 19, 2023
Mini Stable Diffusion. How-to guide? Beginners	0	897	February 15, 2023
Class-Conditional using Unet2DModel 🧨 Diffusers	0	264	October 12, 2023
Unet1dmodel for latent image diffusion 🧨 Diffusers	0	265	April 11, 2023
SDXL img2img controlnet not in release 0.20.2? 🧨 Diffusers	1	499	September 4, 2023

What are the fir and sde_vp kernels in ResnetBlock2D

Related Topics