Masked Language Modeling (MLM) using TFBertForMaskedLM (Tensorflow)

🤗Transformers
1

Where can I find a complete example on how to fine tune a model using Tensorflow for TFBertForMaskedLM for custom text dataset using transformers and Tensorflow instead of PyTorch. I have an working example using PyTorch but I need an example with Tensorflow. Can someone guide me here?

1 Like
2

Looking for the same thing! Have you found a good example by now?

1 Like
3

Hi,

I am also looking for the same thing. Any luck finding more information about this?
Thanks,
Ayala

4

You can create tf records using any data. Have a look at Bert official repo. Then pass tf records to TFBert. But yes it’s not straight forward as PyTorch.

5

Thanks! I will take a look.