Trainer API to log both Training and Validation Metrics

I am fine-tuning for a classification task - I am trying to replicate (and potentially replace) my native PyTorch training and evaluation loops with the Trainer API.

I usually log metrics for both training and validation across each batch/epoch.

Here is what I can achieve with Trainer API.

The accuracy and F1 are of validation sets and I want to also see the same set of metrics at each step for training data as well. Can someone guide on how I can achieve this?


This is not implemented in the Trainer. You can manually evaluate yourself at the end of training on any dataset you want (training set included).

Thanks for the reply @sgugger . Since loss may not be directly related to certain metrics, I was hoping to see where the metrics trend starts overfitting and make decisions based on that. Do you think this is a valid use case to think about adding this to the API in the future?

1 Like