Hi, I’m trying to fine tune SegFormer for binary semantic segmentation. I looked over some quick tutorials like semantic segmentation and fine-tune-segformer but I could not run my case.

I have two classes that are car and background.

```
id2label = { 0: 'background', 1: 'car' }
label2id = { 'background':0, 'car':1}
num_labels = len(id2label)
```

First, I got the **size mismatch** error when I try giving input id2label and label2id to the model variable. Then I convert the ignore_mismatched_sizes=True (I do not know that it is a proper way)

```
model = SegformerForSemanticSegmentation.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512", id2label=id2label, label2id=label2id, ignore_mismatched_sizes=True)
```

Then when I try to train the model with mean_iou metric from evaluate library, I got nan values for all metrics with these warnings:

```
/root/.cache/huggingface/modules/evaluate_modules/metrics/evaluate-metric--mean_iou/08bc20f4f895f3caf75fb9e3fada1404bded3c3265243d05327cbb3b9326ffe9/mean_iou.py:258: RuntimeWarning: invalid value encountered in double_scalars
all_acc = total_area_intersect.sum() / total_area_label.sum()
/root/.cache/huggingface/modules/evaluate_modules/metrics/evaluate-metric--mean_iou/08bc20f4f895f3caf75fb9e3fada1404bded3c3265243d05327cbb3b9326ffe9/mean_iou.py:259: RuntimeWarning: invalid value encountered in divide
iou = total_area_intersect / total_area_union
/root/.cache/huggingface/modules/evaluate_modules/metrics/evaluate-metric--mean_iou/08bc20f4f895f3caf75fb9e3fada1404bded3c3265243d05327cbb3b9326ffe9/mean_iou.py:260: RuntimeWarning: invalid value encountered in divide
acc = total_area_intersect / total_area_label
/root/.cache/huggingface/modules/evaluate_modules/metrics/evaluate-metric--mean_iou/08bc20f4f895f3caf75fb9e3fada1404bded3c3265243d05327cbb3b9326ffe9/mean_iou.py:262: RuntimeWarning: Mean of empty slice
metrics["mean_iou"] = np.nanmean(iou)
/root/.cache/huggingface/modules/evaluate_modules/metrics/evaluate-metric--mean_iou/08bc20f4f895f3caf75fb9e3fada1404bded3c3265243d05327cbb3b9326ffe9/mean_iou.py:263: RuntimeWarning: Mean of empty slice
metrics["mean_accuracy"] = np.nanmean(acc)
Trainer is attempting to log a value of "[nan]" of type <class 'list'> for key "eval/per_category_iou" as a scalar. This invocation of Tensorboard's writer.add_scalar() is incorrect so we dropped this attribute.
Trainer is attempting to log a value of "[nan]" of type <class 'list'> for key "eval/per_category_accuracy" as a scalar. This invocation of Tensorboard's writer.add_scalar() is incorrect so we dropped this attribute.
```

I use the same compute_metrics function as second link.

So, I could not train the model for my scenario and I got some points that I could not figured out.

- I’m not sure what my num_labels parameter should be, 1 or 2.
- SegFormer image processor turn the masks (labels) into 0 or 255 values. Why is it not 0-1. Wouldn’t it be easier for classification?
- There is a parameter called
**ignore_index**in metrics.compute part. I could not understand very well, is it possible to be related the errors? - In the first link, they are using nvidia/mit-b0 for segformer fine tuning but when I check the models on hugginface/models, it is used for image classification and there is seperate
**nvidia/segformer-b0-finetuned-ade-512-512**. What is the reason that they did not used directly segformer-b0? - If segformer is not suitable for binary segmentation, what else can I used for this case (better to be robust ones)?
- I did not find any configuration for loss function also. Is there anything about this I have to do?

So, I’m struggling with this case for 2 days and I need some help . What should I do to use segformer in binary segmentation cases? Thanks in advance