Struggling to transfer 2D cartoon style while keeping face likeness

[Nemayme]

Hello everyone,

I’m building a mobile app for hair salon bookings and need to create a stylist selection carousel. I want consistent 2D cartoon avatars that resemble actual stylists.

My goal: Take a person’s face reference + a style reference image (2D cartoon) and combine them.

My struggle:

• I’m not experienced with ComfyUI, which might be part of the problem

• Followed ChatGPT advice through endless rabbit holes (lost a couple of days like this)

• Tried InvokeAI, training mini LoRAs, ComfyUI IP-Adapter

• Battled compatibility issues and errors

I have two key references:

1. A person’s photo (for likeness)

2. A 2D cartoon style image (generated from my selfie by an online AI service)

I need to apply the cartoon style from reference #2 to the face in reference #1. The style image was created from my selfie, but now I need to use that same style for other people.

What would you do? Is there a straightforward workflow to combine face likeness from one image with artistic style from another? I’m open to any tools or approaches that actually work.

TLDR: New to ComfyUI. Need help combining face reference (person A) with style reference (2D cartoon of person B) to create consistent avatars. Failed with IP-Adapter/LoRAs.

[John6666]

When trying to mimic the style of an image, it’s quite challenging if there are few reference images. In this case, having only one reference image is a significant constraint. This essentially rules out approaches involving LoRA fine-tuning.

It seems to have failed once, but approaches using IP-Adapter variants, ControlNet, or dedicated models are probably more realistic. (The sample script produces output like this.

It’s not great, but it serves as an example.)

Alternatively, if you can prepare many 2D cartoon reference style images, training LoRA is straightforward.