EvalPrediction has an unequal number of label_ids and predictions 😫

The EvalPrediction object received in Trainers compute_metrics function returns an unequal number of label_ids and predictions.

My compute_metrics function given to the Trainer looks like this:

The shapes I print:

And the error message I get:

Does anyone of you have a solution to this? :hugs:

What does your data look like? I’m afraid there might have been some labels that were a list concatenated with single labels or something like that.

I found a solution to the problem!

I struggled with doing multi-label classification using the transformers Trainer until I found the following modification of the Trainer class here: Trainer — transformers 4.11.3 documentation


For some reason I also modified the Trainer for a multi class classification problem I have:

First, I discovered that there is no need for doing that, because the Trainer already does multi class by default :joy:

The problem seemed to be caused by my multi-class Trainer modification :sweat_smile: