How to add multiple metrics to Huggingface Transformers Trainer?

🤗Transformers
1

Is there a simple way to add multiple metrics to the Trainer feature in Huggingface Transformers library?

Here is the code I am trying to use:
from datasets import load_metric
import numpy as np

def compute_metrics(eval_pred):
metric1 = load_metric(“precision”)
metric2 = load_metric(“recall”)
metric3 = load_metric(“f1”)
metric4 = load_metric(“accuracy”)

logits, labels = eval_pred
predictions = np.argmax(logits, axis=-1)
precision = metric1.compute(predictions=predictions, references=labels, average="weighted")["precision"]
recall = metric2.compute(predictions=predictions, references=labels, average="weighted")["recall"]
f1 = metric3.compute(predictions=predictions, references=labels, average="weighted")["f1"]
accuracy = metric4.compute(predictions=predictions, references=labels, average="weighted")["accuracy"]

return {"precision": precision, "recall": recall, "f1": f1, "accuracy": accuracy}

I added the average=‘weighted’ to each compute() because I got this error previously:
target is multiclass but average=‘binary’. please choose another average setting, one of [none, ‘micro’, ‘macro’, ‘weighted’].

However after adding average=‘weighted’, I am getting this error in the middle of training:
_compute() got an unexpected keyword argument ‘average’

Is there a better way to add multiple metrics? The model I am using is google/vit-base-patch16-224-in21k for image classification.

2

One of your metrics seems to not take the average argument, nothing linked to the Trainer :slight_smile: