How do I finetune Blip2 model on a custom dataset?

[sashika]

I am new to this area. I want to fine-tune the blip2 model on a custom dataset. I tried the freezing vision model and the language model but I didn’t get satisfactory results. Do I need to fine-tune the language model? I am working with medical data.

[John6666]

I don’t know much about it, but we have people who know the best about it.

BLIP-2

@nielsr Looks like a question about BLIP2.