Can not reproduce result when finetune BERT model

🤗Transformers
1

I am finetuning BERT model for text classification task and I use Trainer interface to do that. I have already set seed like

random.seed(42)
np.random.seed(42)
torch.manual_seed(42)
torch.cuda.manual_seed(42)
torch.cuda.manual_seed_all(42)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

but the results are still varied between different run sessions. I also have download code from “transformers/examples/pytorch/text-classification/run_classification.py at main · huggingface/transformers · GitHub” and run but I still encounter similar situations. Any idea about this?

1 Like
2

Maybe this?

3

Sorry, after two days debug, I finally find that the problem is my input data different among sessions because order of item in set() is different by hashing mechanism :(((

1 Like
4

lol But glad it was resolved

1 Like