This page shows how to use a custom trainer
from torch import nn from transformers import Trainer class CustomTrainer(Trainer): def compute_loss(self, model, inputs, return_outputs=False): labels = inputs.get("labels") # forward pass outputs = model(**inputs) logits = outputs.get("logits") # compute custom loss (suppose one has 3 labels with different weights) loss_fct = nn.CrossEntropyLoss(weight=torch.tensor([1.0, 2.0, 3.0])) loss = loss_fct(logits.view(-1, self.model.config.num_labels), labels.view(-1)) return (loss, outputs) if return_outputs else loss
My questions -
logits = outputs.get("logits")returns logits then why outputs are always in range 0 to 1? Am I missing something?
- I have class imbalance problem and I would like to use bce with logits loss? That loss requires input to be logits (values before converting to probability) . what should I do to use that loss?