Empty BERT Model, any help?

I am trying to train BERT For Masked LM from the BertForMaskedLM model.
My question is:
Is this model “empty”?

Let me explain further:
With BertForMaskedLM.from_pretrained("bert-base-uncased") we load the “bert-base-uncased” model and can use it.
But if I call the model like this.

model_config = BertConfig(
        vocab_size = vocab_size, 
        max_position_embeddings = max_length

model = BertForMaskedLM( config=model_config )

Am I actually loading BertForMaskedLM without using the pretrained model? Is this model “empty” as i can assume?

The way you instantiate the model does not load the pretrained weights .So, you would be correct in saying it’s “empty”.

1 Like

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.