ValueError: The model did not return a loss from the inputs, only the following keys: logits. For reference, the inputs it received are input_ids,attention_mask

🤗Transformers
1

Hi, I am following this tutorial to fine tune a distilbert-base-uncased model

I don’t use the imdb dataset as in the tutorial but my own review.csv file which has column ‘Text’ and ‘Score’

When I train, it gives the error:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <module>:1                                                                                    │
│                                                                                                  │
│ ❱ 1 trainer.train()                                                                              │
│   2                                                                                              │
│   3                                                                                              │
│                                                                                                  │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:1664 in train                    │
│                                                                                                  │
│   1661 │   │   inner_training_loop = find_executable_batch_size(                                 │
│   1662 │   │   │   self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size  │
│   1663 │   │   )                                                                                 │
│ ❱ 1664 │   │   return inner_training_loop(                                                       │
│   1665 │   │   │   args=args,                                                                    │
│   1666 │   │   │   resume_from_checkpoint=resume_from_checkpoint,                                │
│   1667 │   │   │   trial=trial,                                                                  │
│                                                                                                  │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:1940 in _inner_training_loop     │
│                                                                                                  │
│   1937 │   │   │   │   │   with model.no_sync():                                                 │
│   1938 │   │   │   │   │   │   tr_loss_step = self.training_step(model, inputs)                  │
│   1939 │   │   │   │   else:                                                                     │
│ ❱ 1940 │   │   │   │   │   tr_loss_step = self.training_step(model, inputs)                      │
│   1941 │   │   │   │                                                                             │
│   1942 │   │   │   │   if (                                                                      │
│   1943 │   │   │   │   │   args.logging_nan_inf_filter                                           │
│                                                                                                  │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:2735 in training_step            │
│                                                                                                  │
│   2732 │   │   │   return loss_mb.reduce_mean().detach().to(self.args.device)                    │
│   2733 │   │                                                                                     │
│   2734 │   │   with self.compute_loss_context_manager():                                         │
│ ❱ 2735 │   │   │   loss = self.compute_loss(model, inputs)                                       │
│   2736 │   │                                                                                     │
│   2737 │   │   if self.args.n_gpu > 1:                                                           │
│   2738 │   │   │   loss = loss.mean()  # mean() to average on multi-gpu parallel training        │
│                                                                                                  │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:2780 in compute_loss             │
│                                                                                                  │
│   2777 │   │   │   │   loss = self.label_smoother(outputs, labels)                               │
│   2778 │   │   else:                                                                             │
│   2779 │   │   │   if isinstance(outputs, dict) and "loss" not in outputs:                       │
│ ❱ 2780 │   │   │   │   raise ValueError(                                                         │
│   2781 │   │   │   │   │   "The model did not return a loss from the inputs, only the following  │
│   2782 │   │   │   │   │   f"{','.join(outputs.keys())}. For reference, the inputs it received   │
│   2783 │   │   │   │   )                                                                         │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: The model did not return a loss from the inputs, only the following keys: logits. For reference, the 
inputs it received are input_ids,attention_mask.

Please help

2

Are you still trying to resolve this? I used the IMDB dataset from the tutorial and its features are [‘text’, ‘label’, ‘input_ids’, ‘attention_mask’] after the pre-processing step. It seems you may not have the same features in your custom dataset?

I get the same error in a different context that I’m trying to resolve, so if fixing the columns doesn’t help, would be curious to try it with a snippet of your dataset.