I am trying to fine-tune Stable Diffusion 2 for a particular use case. I have a style I am trying to emulate under a variety of settings. The goal is to fine-tune Stable Diffusion 2 to produce variations on the settings I’m training within the style while leveraging Dreambooth style techniques to avoid catastrophic forgetting.
I’m having a hard time figuring out where to start here leveraging the hugging face ecosystem? I haven’t found any clear examples on how to make a custom training regime like this leveraging an existing checkpoint.
Any pointers on how to do this would be appreciated.