Hello,
I’m interested in pre-training the Albert model on a large set of domain specific data. I see on GitHub that there is a TFAlbertForPreTraining
class, but it is not showing up in the docs.
Is this class only there for the loading of pre-trained Albert models, or can I make use of this to do pre-training myself?