Fixing the random seed in the Trainer does not produce the same results across runs

For full reproducibility you need to instantiate your model inside the Trainer by using the model_init argument (or setting a seed before instantiating your model). You have random weights in your model head and those are different in your two runs in the code you show.