I have been playing around with the DreamBooth example in Colab (Google Colab) and it’s awesome. However, I came to wonder if there’s a way to add more than one person into the same model.
I did some testing, and I think that what is preventing me from making it work is not knowing what should be on the path of pretrained_model_name_or_path (from train_dreambooth.py). I naively tried $OUTPUT_DIR and as it didn’t seem to work, I created a .ckpt in the same folder and gave a shot with pretrained_model_name_or_path=$OUTPUT_DIR_artista+“/model.ckpt”, but also didn’t had any luck. So, if someone could help me with this, I would be pretty thankful 
Okay, I came up with a solution, so I’ll share it in case someone needs it in the future.
Right after the first fine tuning, you got to run this:
StableDiffusionPipeline.from_pretrained(OUTPUT_DIR, torch_dtype=torch.float16).save_pretrained(OUTPUT_DIR)
Then you can use DreamBooth again with pretrained_model_name_or_path=$OUTPUT_DIR.
However, I noticed something that other people have already pointed out about using DreamBooth more than once. Each time you use it, all the model is adjusted to generate the current instance, so it not only loses generalization, but it also gets worse at producing the instances previously added with DreamBooth. I read that an alternative is to use Textual Inversion with DreamBooth, but I haven’t tested it yet.
1 Like