Issue with converting my own BERT TF2 checkpoint to PyTorch and loading the PyTorch model for training

🤗Transformers
1

Hi,

I’m using huggingface to train my own bert model. I have checkpoints which are in TensorFlow2 and I have converted them successfully in PyTorch. The checkpoint conversion script has created .bin file which is having following subdirectories and pkl file.

Screenshot from 2021-02-25 17-00-20

I wanted to gain some more information on the following points:

  • Am’I doing the conversation of TensorFlow checkpoint to PyTorch checkpoint correct? I can see the pre-trained bert-base-uncased model which is hosted on hugging face model repository is just having single pytorch_model.bin file.

  • How I can load this custom BERT-PyTorch model’s checkpoint using modeling_bert.py script?

Any help or suggestions will be helpful.

Thanks