Trainer gives error after 1st epoch and evaluation

🤗Transformers
1

Hi I have managed to get my code working for training, yet after 1 epoch I am getting weird results for the evaluation and also an error

Trainer is attempting to log a value of "[0. 0. 0. 1. 0.]" of type 
<class 'numpy.ndarray'> for key "eval/recall" as a scalar. This 
invocation of Tensorboard's writer.add_scalar() is incorrect 
so we dropped this attribute.

and then an error

TypeError: Object of type ndarray is not JSON serializable

I used the compute_metrics (measure accuracy, f1, precision and accuracy) provided by the tutorial and I am doing a classification task with 5 task.

1 Like
2

I used the compute_metrics (measure accuracy, f1, precision and accuracy) provided by the tutorial and I am doing a classification task with 5 task.

Could you tell us which tutorial? From the error message, it seems your metric value is a NumPy array, so sharing your compute_metric function would also help.

3

Of course. it is from the Training and Finetuning tutorial and this is my code:

# Metrics
from sklearn.metrics import accuracy_score, precision_recall_fscore_support

def compute_metrics(pred):
    labels = pred.label_ids
    preds = pred.predictions.argmax(-1)
   precision, recall, f1, _ = precision_recall_fscore_support(labels, preds)
   acc = accuracy_score(labels, preds)
   return {
        'accuracy': acc,
        'f1': f1,
        'precision': precision,
       'recall': recall
    }
4

precision_recall_fscore_support requires an argument average to return simple numbers (and not arrays). See the doc to choose the one that suits your need here.

1 Like
5

I ran into an issue like this when I inadvertently logged a metric in the trainer that was a Tensor instead of just a float. I did something like average_scores.mean(), which returned a Tensor object, which was somehow affixed to Trainer.state and then broke json-saving. I changed the metric computation to average_scores.mean().item() and this fixed the issue.