I am interested in training a language model from scratch. Not fine tuning the usual distilbert ---- running the whole thing on my GPU instead !
I found this interesting notebook How to train a new language model from scratch using Transformers and Tokenizers and I would be interested to know if there is one that uses
tensorflow instead. I cannot have
pytorch on my machine unfortunately.
Is there a huggingface example notebook that would help me do that?