The model did not return a loss from the inputs, only the following keys: logits. For reference, the inputs it received are input_values

Intermediate
3

I get a similar error when using

# Set DistilBERT tokenizer
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

# Define DistilBERT as our base model:
model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased", num_labels=9)

# Use data_collector to convert our samples to PyTorch tensors and concatenate them with the correct amount of padding
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

Specifically, when using

# Define a new Trainer with all the objects we constructed so far
repo_name = "sentiment-model-amazon-reviews-distilbert"

training_args = TrainingArguments(
    output_dir=repo_name,
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=1,
    weight_decay=0.01,
    save_strategy="epoch", 
    push_to_hub=True
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_train,
    eval_dataset=tokenized_cal,
    tokenizer=tokenizer,
    data_collator=data_collator,
    compute_metrics=compute_metrics
)

# Train and push to hub
trainer.train()

returns

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

<ipython-input-50-f0acdc25090a> in <module>
     24 
     25 # Train and push to hub
---> 26 trainer.train()

3 frames

/usr/local/lib/python3.8/dist-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1541             self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size
   1542         )
-> 1543         return inner_training_loop(
   1544             args=args,
   1545             resume_from_checkpoint=resume_from_checkpoint,

/usr/local/lib/python3.8/dist-packages/transformers/trainer.py in _inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   1789                         tr_loss_step = self.training_step(model, inputs)
   1790                 else:
-> 1791                     tr_loss_step = self.training_step(model, inputs)
   1792 
   1793                 if (

/usr/local/lib/python3.8/dist-packages/transformers/trainer.py in training_step(self, model, inputs)
   2537 
   2538         with self.compute_loss_context_manager():
-> 2539             loss = self.compute_loss(model, inputs)
   2540 
   2541         if self.args.n_gpu > 1:

/usr/local/lib/python3.8/dist-packages/transformers/trainer.py in compute_loss(self, model, inputs, return_outputs)
   2582         else:
   2583             if isinstance(outputs, dict) and "loss" not in outputs:
-> 2584                 raise ValueError(
   2585                     "The model did not return a loss from the inputs, only the following keys: "
   2586                     f"{','.join(outputs.keys())}. For reference, the inputs it received are {','.join(inputs.keys())}."

ValueError: The model did not return a loss from the inputs, only the following keys: logits. For reference, the inputs it received are input_ids,attention_mask.
1 Like