Trainer API to log both Training and Validation Metrics

🤗Transformers
1

I am fine-tuning for a classification task - I am trying to replicate (and potentially replace) my native PyTorch training and evaluation loops with the Trainer API.

I usually log metrics for both training and validation across each batch/epoch.

Here is what I can achieve with Trainer API.
image

The accuracy and F1 are of validation sets and I want to also see the same set of metrics at each step for training data as well. Can someone guide on how I can achieve this?

Thanks.

2

This is not implemented in the Trainer. You can manually evaluate yourself at the end of training on any dataset you want (training set included).

3

Thanks for the reply @sgugger . Since loss may not be directly related to certain metrics, I was hoping to see where the metrics trend starts overfitting and make decisions based on that. Do you think this is a valid use case to think about adding this to the API in the future?

1 Like