MobileViT Fine tuning

[jpjailson]

Hi,

I’m fine-tuning MobileViT for binary semantic segmentation using a custom dataset with two classes: background and polyp.

I’ve followed several tutorials, including:

• Image Segmentation
• Fine-Tuning a Semantic Segmentation Model on a Custom Dataset and Usage via the Inference API

However, my model achieves high accuracy and IoU only for the background class, while the polyp class consistently gets 0 for both IoU and accuracy.

I’ve already tried setting ignore_index=0 in metric.compute, but the issue persists. Any insights on what might be causing this and how to resolve it?

Thanks in advance!