Fine Tuning Segformer on Custom Dataset, getting negative loss

I am fine tuning a segformer model from Hugging Face on my custom dataset, I am seeing negative losses. The predictions were completely off too. What am I missing?

I am following this blog: Fine-Tune a Semantic Segmentation Model with a Custom Dataset

Related question, where is the mapping for annotation’s color to the label is supposed to be done.
In my case there is just 1 label whose pixel value in the mask image is 254 (gray scale) and everything else is 100. However, how does the model know what is what?


Were you able to solve this problem. I am facing the same issue.

Nikhil Masinete