How to Inpaint with Stable Diffusion 3 Medium?

I need an inpaint model, to ‘guess’ what should be in the dark box on the image.

I found a model runwayml/stable-diffusion-inpainting and as I understood it was fine-tuned for inpaint task. But I guess SD3 would generate better results than old models.

I wonder how to use sd3 for inpaint task ? And if there is any way to fine-tune it in case of bad results but better than sd-inpainting model ?

I was looking for similar information at internet but found only guides how to use pretrained inpainting models.

do you get the model? I also want to get sd3 for inpaint model

Still have no information on its possibility

Checkout the current develop version diffusers@0.30.0.dev0 there is a StableDiffusion3InpaintPipeline available

1 Like

To use SD3 for inpainting, start by adapting it with relevant training data and settings suitable for inpainting tasks. Fine-tuning involves adjusting training parameters and datasets to optimize results beyond the capabilities of older models like SD-inpainting. Explore training methodologies and adapt the model iteratively based on performance evaluations to achieve better inpainting outcomes.

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.