Difference between using or not compute_metric

Beginners
1

Hello, I am newbie in HF and I found there are two different implementations as follows:

def compute_metrics(eval_pred):
logits, labels = eval_pred
predictions = np.argmax(logits, axis=-1)
return metric.compute(predictions=predictions, references=labels)

In train API, some codes call compute_metrics while some does not. What is the differnce between using and no using this metrics function. Can you please explain how it can effect the system predictions??

trainer = Trainer(
model=model,
args=training_args,
train_dataset=dataset_train,
eval_dataset=dataset_test,
#compute_metrics=compute_metrics,
callbacks=[EarlyStoppingCallback(early_stopping_patience=1)]
)

2

Compute metrics is to help track how the training is performing. It does not affect the training itself.
So think of it is a reporting function that calculates extended metrics to help you evaluate the quality of training. You can then use the metrics it reports to fine tune the training.

see this example here.

1 Like
3

thank you so much