Unable to set num_labels=1 for TFRoberta for sequence classification

🤗Transformers
1

I am trying to use this family of sentiment models to do my own classification. However, I have tried to set this up as a regression model, where the score can be a float [0, 1,0] (although i could still cast this as a binary classification if i want).

I found that it isn’t generally possible to use num_labels argument in from_pretrained.

TFRobertaForSequenceClassification.from_pretrained("cardiffnlp/twitter-roberta-base-emotion", num_labels=1)

This will give ValueError: cannot reshape array of size 3072 into shape (768,1).

Is this caused by some specifics of the architecture of roberta? such that it is not straightforward to configure the appropriate classification head? At the API level, this seems to work for the Bert Families of models.

2

I seemed to find a way, after digging into the code a little bit. I am not sure what the caveats are, if this is too hackish

model = TFAutoModelForSequenceClassification.from_pretrained("siebert/sentiment-roberta-large-english")
config = model.config
config.num_labels = 1

classifier = TFRobertaClassificationHead(config, name="classifier")

model.classifier = classifier