Can't carry out LoRa training for Interior Design

Beginners
1

Hi, everyone!

I hope you’re all doing well! :blush:

I’m working on an AI-powered interior design app, and I’m running into some issues while trying to set up LoRA training for Stable Diffusion XL (SDXL). I’m using ChatGPT-3 and Gemini CLI for help, but it seems like I keep hitting roadblocks.

Here’s a bit about what I’ve tried:

  1. On RunPod:
  • I tried using the Koya template for LoRA to fine-tune SDXL, but I kept running into problems, especially with missing configuration files like model_index.json.
  1. On Google Colab:
  • I mounted Google Drive and made sure everything was in place, but I’m still having trouble loading the SDXL model. Even though I tried downloading it directly from Hugging Face, I couldn’t get everything to work correctly.

I’m not a programmer, so I’m really looking for any guidance or advice from people who have worked with SDXL, LoRA, or similar training setups. If anyone has any tips on what I might be missing or how to fix these issues, I would really appreciate it! :folded_hands:

Thanks so much for any help, I really appreciate it!

1 Like
2

I think you are probably using a single safetensors file of about 7GB as the model file, but that is a model file for A1111 WebUI or ComfyUI, not the file format used for training with Diffusers. Although they appear similar at first glance, they are not compatible.The model file for Diffusers is split into multiple JSON configuration files, folders, and a safetensors file. Conversion between formats is possible and may occur automatically, but if a pre-converted file is available, it’s probably best to use that.

If you’re new to training with LoRA, I recommend finding an article with practical tips on creating LoRA and following it step by step. Once you have a general understanding, you can delegate the code generation to a generative AI tool.

sdxla1111diffusers
sdxla1111diffusers1590×948 136 KB

3

Thank you John so much!

1 Like