Additional loss logging

🤗Transformers
1

Hi there,

I’ve overridden the compute_loss method in order to add additional kl_div loss for distillation. Now, I want to log the cross-entropy and KL divergence along with the total loss. I’m wondering if there’s an easy way to do this without overriding about half of the Trainer class?

Thank you in advance!

1 Like
2

I created a way to do that without modifying too much of the trainer class.

You can find my workaround here:

https://github.com/naba89/custom_hf_trainer