【Solved】How can I get loss by using trainer when training gpt2?

[Azily]

I want to use trainer to train my gpt-2 model, while when I pass the dataset to the trainer, it tell me that there is a keyError: ‘loss’. So I try to find the cause and find that the function ‘compute_loss’ is not work as what I want. So I write a new compute_loss function but find that the inputs don’t have the column ‘labels’, which I create when using tokenizer, I think is may be deleted when used by trainer.

I wonder how should I use trainer correctly so that I can make compute_loss work well

this is my compute_loss function, but not work because it does not find a column named labels

[Azily]

there are more pictures about the problem
this is my dataset, it is only used to test how to use trainer so I don’t distinguish them

[Azily]

this is my trainer

[Azily]

By using GPT2LMHeadModel instead of GPT2Model, the problem doesn’t appear again. So I wonder what is the difference between them.