Unclip fine tuning

I’d like to fine-tune stabilityai/stable-diffusion-2-1-unclip at main but the repo has a bunch of models, each with their own config.json. It seems at the very least I’d want to fine tune the text and image encoders but it’s not clear to me how.