I have an unbalanced data with a couple of classes with relatively smaller sample sizes. I am wondering if there is a way to assign the class weights to BertFor SequenceClassification class, maybe in BertConfig ?, as we can do in nn.CrossEntropyLoss. Thank you, in advance!

Class weights for bertForSequenceClassification

sgugger October 21, 2020, 2:34pm 2

No, you need to compute the loss outside of the model for this. If you’re using Trainer, see here on how to change the loss form the default computed by the model.

Topic		Replies	Views
Which loss function in bertforsequenceclassification regression Beginners	7	13021	February 25, 2021
Training with class weights 🤗Transformers	5	759	November 18, 2023
Metrics mismatch between BertForSequenceClassification Class and my custom Bert Classification Beginners	3	755	December 10, 2020
TypeError: cross_entropy_loss(): argument 'input' (position 1) must be Tensor, not SequenceClassifierOutput 🤗Transformers	2	5744	April 26, 2022
How do i take only "BERT" weights from BertForSequenceClassification model? 🤗Transformers	0	1259	February 16, 2022

Class weights for bertForSequenceClassification

Related Topics