I try to rebuild the TFBertForSequenceClassification model from the plain TFBertModel.
TFBertForSequenceClassification uses a dropout and a dense layer on top of the main BERT model. The number of neurons is config.num_labels.
I wonder, where is this values because it seems it has no default value.
And finetuning TFBertForSequenceClassification like here:
does not handle a value for num_labels.
So where does the 2 neurons come from in the last layer (by default)?
self.num_labels = config.num_labels
but config.num_labels has no default value?
No it has no default value because it is supposed to be passed at init either because:
- the user is loading a pretrained model and its config has some
- the user is passes
num_labels=n when using a generic pretrained model and it udpates the config.
But as I posted I did not see where the value in the config is set… Where can I look what value is set?
If you’re loading a pretrained model that has been finetuned for a classification task, you can look at the value in its config file with:
from transformers import AutoConfig
config = AutoConfig.from_pretrained('checkpoint_name')