Masked Language Modeling (MLM) using TFBertForMaskedLM (Tensorflow)

Where can I find a complete example on how to fine tune a model using Tensorflow for TFBertForMaskedLM for custom text dataset using transformers and Tensorflow instead of PyTorch. I have an working example using PyTorch but I need an example with Tensorflow. Can someone guide me here?

1 Like

Looking for the same thing! Have you found a good example by now?

1 Like


I am also looking for the same thing. Any luck finding more information about this?

You can create tf records using any data. Have a look at Bert official repo. Then pass tf records to TFBert. But yes it’s not straight forward as PyTorch.

Thanks! I will take a look.