So I trained a controlnet on input with more than 3 channels. The model trained and the validation logger worked perfectyl, I did have to change one or two things in the source code but nothing major. But I cant get ControlNet.from_pretrained to work.
Blockquote /usr/local/lib/python3.10/dist-packages/accelerate/utils/modeling.py in
value, dtype, fp16_statistics)
279 if value is not None:
280 if old_value.shape != value.shape:
→ 281 raise ValueError(
282 f’Trying to set a tensor of shape {value.shape} in “{tensor_name}” (which has shape {old_value.shape}), this look incorrect.’
283 )
ValueError: Trying to set a tensor of shape torch.Size([16, 10, 3, 3]) in “weight” (which has shape torch.Size([16, 3, 3, 3])), this look incorrect.
I have set conditioning_in channels in controlnet.py to 10 in the diffusers source file but not sure what more to do.