Two labels, getting converted to 0 and 247

Someone please help me out here, while doing augmentation with SegformerImageProcessor from Transformers, my original labels are 0 for background and 1 for object, I’m seeing the values in my labels are of two individual values 255 and 247, and 247 is out of bunds ERROR. 255 because i set reduce_labels to true, and 0 gets converted to 255 is what the doc says. I converted my labels to numpy array before augmentation and it has the values 0 and 248, but on another scenario where i had used segments.ai, the labels after being passed to SegformerImageProcessor was returning values 0 and 1, which was as expected, my doubt is can i manually convert the labels 255 to 0 and 247 to 1 in the first case, and hopefully trainer.train() will run then without errors, Or is my approach wrong? Or should I make any changes to my labels dataset to begin with.