Hi,
I upgraded transformers
from v4.9
to v4.32
and now the trainer doesn’t get into my compute_metrics
function. Nothing has changed in the code besides the upgrade.
Debugging resulted in finding the place the compute_metrics
is being skipped- trainer.evaluation_loop()
, line 3255:
# Metrics!
if self.compute_metrics is not None and all_preds is not None and all_labels is not None:
if args.include_inputs_for_metrics:
metrics = self.compute_metrics(
EvalPrediction(predictions=all_preds, label_ids=all_labels, inputs=all_inputs)
)
else:
metrics = self.compute_metrics(EvalPrediction(predictions=all_preds, label_ids=all_labels))
else:
metrics = {}
The if condition is False because all_labels=None
. I think its because of the prediction_step()
that happens before, which returns labels=None
and loss=None
.
Would appreciate any help with that!
Thanks