Segformer train doesn't recognize downsized images

In the past I have used segformer cityscapes 1024 pretrained model but now I am using adek512 b0 model for Semantic segmentation with 1024x1024 images it worked but if I use 416x416 images the train model trows several errors with the rescale or cuda or bool data when I tried to resize to 256 or 512. What would be the right parameters for feature_extractor? Do I need to use another fine running?