Empty BERT Model, any help?

Beginners
1

I am trying to train BERT For Masked LM from the BertForMaskedLM model.
My question is:
Is this model “empty”?

Let me explain further:
With BertForMaskedLM.from_pretrained("bert-base-uncased") we load the “bert-base-uncased” model and can use it.
But if I call the model like this.


# INITIALIZE THE MODEL
model_config = BertConfig(
        vocab_size = vocab_size, 
        max_position_embeddings = max_length
      )

model = BertForMaskedLM( config=model_config )

Am I actually loading BertForMaskedLM without using the pretrained model? Is this model “empty” as i can assume?

2

bert
bert1017×321 31.4 KB

The way you instantiate the model does not load the pretrained weights .So, you would be correct in saying it’s “empty”.

1 Like
3

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.