Class weights for bertForSequenceClassification

Beginners
3

Hi Sylvain,

Glad to hear from you, outside of FastAI :slight_smile: Well, I am here as “Beginner” and will have to study more about Trainer. In the meantime, I tried the following:

  1. run BertForSequenceClassification as usual
  2. Take out logits from output (discard the loss from Bert run)
  3. calculate new loss from nn.CrossEntropyLoss
  4. and then calculate loss.backward()

Model runs okay, but I am not sure if this is a legitimate approach…

Thanks.

2 Likes