BertForSequenceClassification can be used for regression when number of classes is set to 1. The documentation says that BertForSequenceClassification calculates cross-entropy loss for classification. What kind of loss does it return for regression? (I’ve been assuming it is root mean square erro…

This is the GitHub link At line 1354, you have the condition to check the labels (if it is one or more) if self.num_labels == 1: # We are doing regression loss_fct = MSELoss() loss = loss_fct(logits.view(-1), labels.view(-1)) else: loss_fct = CrossEntropyLoss() loss = loss_fct(logits.view(-…

Which loss function in bertforsequenceclassification regression

Beginners

Karthik12 October 9, 2020, 9:50am 4

Thanks @BramVanroy, this makes the code easier to read. I will do so, in future.

Topic		Replies	Views
Class weights for bertForSequenceClassification Beginners	10	12728	May 29, 2022
Fine-tune BERT and Camembert for regression problem Beginners	17	11229	December 4, 2021
How to fine-tune Bert on STS-B task? Beginners	2	1438	August 4, 2021
How do I do multi Class (multi head) classification? 🤗Transformers	6	4454	October 18, 2022
How to convert single-class BertForSequenceClassification prediction into probability? Models	0	425	August 4, 2022

Which loss function in bertforsequenceclassification regression

Related topics