KandinskyV22ControlnetPipeline class, where does Controlnet actually come into play?
I am confused because the
UNet2DConditionModel has a parameter
is_controlnet which is set as follows:
is_controlnet = mid_block_additional_residual is not None and down_block_additional_residuals is not None
KandinskyV22ControlnetPipeline neither specifies
down_block_additional_residuals when calling the unet,
is_controlnet is therefore False in the unet. The controlnet hint is consequently simply concatenated to the sample input, but that’s not what controlnet is right?
So my question is: Where is the controlnet? And what is the deal with the
is_controlnet variable in unet?