I tried doing this at the start without luck. I was wondering if anyone was successful at eval/retrain GPT2 with TensorFlow instead of pyTorch? I guess there are some ports directly in TF. However, my question is related to the Transformers library.
I am not aware of public notebooks regarding training Huggingface’s TFGPT2, but we have public notebooks of training TFT5 and TFXLM-R , which should give you some ideas
TF-T5
TF-XLM-R