BertForSequenceClassification can be used for regression when number of classes is set to 1. The documentation says that BertForSequenceClassification calculates cross-entropy loss for classification. What kind of loss does it return for regression? (I’ve been assuming it is root mean square erro…

This is the GitHub link At line 1354, you have the condition to check the labels (if it is one or more) if self.num_labels == 1: # We are doing regression loss_fct = MSELoss() loss = loss_fct(logits.view(-1), labels.view(-1)) else: loss_fct = CrossEntropyLoss() loss = loss_fct(logits.view(-…

Which loss function in bertforsequenceclassification regression

Beginners

rgwatwormhill February 24, 2021, 7:25pm 7

Hi theudster,

the pytorch docs for CrossEntropyLoss suggest that you can add a weight tensor CrossEntropyLoss — PyTorch 1.7.1 documentation . What happens if you try it?

Topic		Replies	Views
Class weights for bertForSequenceClassification Beginners	10	12729	May 29, 2022
Fine-tune BERT and Camembert for regression problem Beginners	17	11229	December 4, 2021
How to fine-tune Bert on STS-B task? Beginners	2	1439	August 4, 2021
How do I do multi Class (multi head) classification? 🤗Transformers	6	4454	October 18, 2022
How to convert single-class BertForSequenceClassification prediction into probability? Models	0	425	August 4, 2022

Which loss function in bertforsequenceclassification regression

Related topics