Reproducible model between SetFit Versions?

Previously I trained a SetFit model using SetFit v0.6.0 (py 3.8)

I have followed the migration steps and have refactored code to move to SetFit v1.0.3 (py 3.11).

I have used the same hyperparameters, and the same random seeds and the same training data.

Output of the v1.0.3 model is reproducible: the same model is created each time.
Output of the v0.6.0 model is also reproducible.

The two models are not the same, and their outputs are significantly different.

Is this expected?

1 Like