Load Bert model weights to transformers v3 from model trained with transformers v2

I have a BertModel (bert-base-uncased) trained with transformers==2.1.1 (using PyTorch v1.3)

However, we want to move to transformers v3. I was wondering if it would be possible to load the model weights of that model into transformers v3. Otherwise, we would have to re-train the language model and we want to avoid doing that.

What would be the best way to do this?

It’s possible, you won’t need to anything special, you should be able to load it using from_pretrained method.

1 Like

Thanks, I figured what the issue was. We were pickling our models.

I had to load the model and write it out to a directory using bert_model.save_pretrained(save_dir) and then load it in v3 using BertModel.from_pretrained(save_dir)