How can I add multi-conditioning to a diffusion model, it can be a text embedding and an image. I tried with concatenation and cross attention. But experiencing a problem, the performance of the model increases to a certain point and keeps on decreasing after that, but the loass curves looks perfect both are decreasing. I tried with earlystopping but teh model is not reaching teh expected performance. I appreciate if there is any lead on this from someone.
1 Like
Hmm… How about using IP-Adapter or T2I-Adapter or ControlNet, etc. ?
Hi John thanks for the response, the model I am using is a DDPM based one Hope it will work for the same
1 Like
I can’t say for sure whether the weight will function as it is now, but in principle, I think something similar should work.
Okay thanks John
1 Like