How to load finetuned model in TF

Beginners
1

So I trained a new gpt2 model using the language modeling script and it spits out a pytorch.bin file that I assume is the model. How do I go about converting it to TF 2.0? I see there is a convert script but I am not sure if I can just run that to make it TF compatible?

2

You should create your model like this:

from transformers import TFGPT2Model
model = TFGPT2Model.from_pretrained("path_to_dir", from_pt=True)

where path_to_dir should be replaced with the path to the directory where your pytorch model is (what you set in GPT2Model.save_pretrained()).

Then you can use your TF model and save it with the save_pretrained method.

1 Like
3

Awesome, I’ll give that try. I missed that from_pt param.