What is default value for number of labels in BertForSequenceClassification?

🤗Transformers
1

Hi,

I try to rebuild the TFBertForSequenceClassification model from the plain TFBertModel.

TFBertForSequenceClassification uses a dropout and a dense layer on top of the main BERT model. The number of neurons is config.num_labels.

I wonder, where is this values because it seems it has no default value.

And finetuning TFBertForSequenceClassification like here:

does not handle a value for num_labels.

So where does the 2 neurons come from in the last layer (by default)?

Look here:

there is

self.num_labels = config.num_labels

but config.num_labels has no default value?

See

2

No it has no default value because it is supposed to be passed at init either because:

  • the user is loading a pretrained model and its config has some num_labels
  • the user is passes num_labels=n when using a generic pretrained model and it udpates the config.
3

But as I posted I did not see where the value in the config is set… Where can I look what value is set?

4

If you’re loading a pretrained model that has been finetuned for a classification task, you can look at the value in its config file with:

from transformers import AutoConfig
config = AutoConfig.from_pretrained('checkpoint_name')
config.num_labels
1 Like