For people interested in tools for logging and comparing different models and training runs in general, Weights & Biases is directly integrated with Transformers.
You just need to have
wandb installed and logged in.
It automatically logs losses, metrics, learning rate, computer ressources, etc.
Here is another cool example where I ran a sweep to fine-tune GPT-2 on tweets.
Finally you can use your runs to create cool reports. See for example my huggingtweets report.
At the moment it is integrated with
If you use Pytorch Lightning, you can use
WandbLogger. See Pytorch Lightning documentation.
Let me know if you have any questions or ideas to make it better!