Load Bert model weights to transformers v3 from model trained with transformers v2

🤗Transformers
1

I have a BertModel (bert-base-uncased) trained with transformers==2.1.1 (using PyTorch v1.3)

However, we want to move to transformers v3. I was wondering if it would be possible to load the model weights of that model into transformers v3. Otherwise, we would have to re-train the language model and we want to avoid doing that.

What would be the best way to do this?

2

It’s possible, you won’t need to anything special, you should be able to load it using from_pretrained method.

1 Like
3

Thanks, I figured what the issue was. We were pickling our models.

I had to load the model and write it out to a directory using bert_model.save_pretrained(save_dir) and then load it in v3 using BertModel.from_pretrained(save_dir)