Thanks @John6666 for your input. I tried and it did not work. They were trying to resolve the model output inconsistency between batch run and single run, but my issue is the model output inconsistency between different transformers versions (4.39.2 vs 4.48.0). Also, the inconsistency lies in the masked portion only, but not in the unmasked portion.
1 Like