Goal: AI generated images of my family in artistic (anime/drawn style) action scenes.
Technical: I have LoRAs of my family’s faces, and obviously any model I want from huggingface. Using this I made two attempts to generate images:
- Use models biased towards drawings to get action images (Good, depending on prompting) + yolo11 face recognition (masks are ok 60% of the time) + inpainting with the lora file of note and the family member’s name with various weights. It’s this last step that fails horribly. I don’t/cant’ get a recognizable face inpainted in the images in question(
self.inpaint_pipe(prompt, negative_prompt, image, mask_image, …, guidance=7.0, strength=0.6)) despite some pretty heavy tuning of / playing with the parameters. Is this a reasonable path or do I overestimate how much can get done here to get drawing style images out of real-world photos/loras without other steps? - Attempt two, separate from 1, is to just combine the lora up with the model and ask for the whole scene. This result varies heavily by the model, but generally I can get high quality boring scenes with recognizable faces (photo realistic) or low quality non-recognizable faces in action anime scenes. The code rather no different than a generic comfyui wiring model→lora + empty latent → ksample → vae decode → save.
How should I approach this problem? I’m looking more for a DIY than a service and the social media forums are all about hocking services it seems.