Xls-r fine tuening does not give the same output when running multiple times

When running xls-r and setting seed=49 on the training argumnets, it does not give the same output on the final WER, sometimes it is 0.11 sometimes its 0.29
What can be the reason?