Finetune Stable Diffusion with depth controlnet on interior datasets

AsmaAILab · June 21, 2025, 4:04pm

I am creating an interior design app, and I am stuck in fine-tuning stable diffusion using controlnet depth condition. But I am stuck on whether I should fine-tune only ControlNet for depth condition or to train Stable Diffusion as well for image generation on my custom dataset.

John6666 · June 22, 2025, 2:57am

Not working well with existing Depth ControlNet?

github.com/R3gm/stablepy

stablepy/diffusers_vanilla/constants.py

main


      
              },
          }
          
          CONTROLNET_MODEL_IDS = {
              "openpose": ["lllyasviel/control_v11p_sd15_openpose", "r3gm/controlnet-openpose-sdxl-1.0-fp16"],
              "canny": ["lllyasviel/control_v11p_sd15_canny", "r3gm/controlnet-canny-scribble-integrated-sdxl-v2-fp16"],
              "mlsd": ["lllyasviel/control_v11p_sd15_mlsd", "r3gm/controlnet-union-sdxl-1.0-fp16"],
              "scribble": ["lllyasviel/control_v11p_sd15_scribble", "r3gm/controlnet-canny-scribble-integrated-sdxl-v2-fp16"],
              "softedge": ["lllyasviel/control_v11p_sd15_softedge", "r3gm/controlnet-union-sdxl-1.0-fp16"],
              "segmentation": ["lllyasviel/control_v11p_sd15_seg", "r3gm/controlnet-union-sdxl-1.0-fp16"],
              "depth": ["lllyasviel/control_v11f1p_sd15_depth", "r3gm/controlnet-union-sdxl-1.0-fp16"],
              "normalbae": ["lllyasviel/control_v11p_sd15_normalbae", "r3gm/controlnet-union-sdxl-1.0-fp16"],
              "lineart": ["lllyasviel/control_v11p_sd15_lineart", "r3gm/controlnet-union-sdxl-1.0-fp16"],
              "lineart_anime": ["lllyasviel/control_v11p_sd15s2_lineart_anime", "r3gm/controlnet-lineart-anime-sdxl-fp16"],
              "shuffle": "lllyasviel/control_v11e_sd15_shuffle",
              "ip2p": "lllyasviel/control_v11e_sd15_ip2p",
              "inpaint": "lllyasviel/control_v11p_sd15_inpaint",
              "txt2img": "Nothinghere",
              "sdxl_canny_t2i": "TencentARC/t2i-adapter-canny-sdxl-1.0",
              "sdxl_sketch_t2i": "TencentARC/t2i-adapter-sketch-sdxl-1.0",
              "sdxl_lineart_t2i": "TencentARC/t2i-adapter-lineart-sdxl-1.0",

Topic		Replies	Views
Multi_controlnet + inpaint 🧨 Diffusers	5	3578	November 12, 2023
Stable Diffusion ControlNet fine-tune quality issue 🧨 Diffusers	0	541	June 4, 2024
I'm curious about the conditioning of the control net 🧨 Diffusers	0	224	January 15, 2024
Image2image SDLX finetuning using python 🧨 Diffusers	3	101	February 25, 2025
Automatic1111 or ComfyUI Beginners	0	245	August 13, 2024

Finetune Stable Diffusion with depth controlnet on interior datasets

Related topics