Here is a Colab notebook demonstrating the issue Google Colab After the initial period of the XLA compilation, training proceeds quickly. When the evaluation rolls around at the end of an epoch it’s extremely slow. I assumed perhaps there was just another initial period of slowness, but after 25 mi…

T5 evaluation via Trainer `predict_with_generate` extremely slow on TPU?

isspek November 2, 2023, 1:27pm 2

+1 I have a similar issue when I fine-tune with GPU. The training takes no long, however predictions on development set takes too long.

Topic		Replies	Views
TPU slow finetuning T5-base Models	13	3070	June 17, 2022
[Urgent] trainer.predict() and model.generate creates totally different predictions 🤗Transformers	4	6933	February 1, 2021
TensorFlow trainer 🤗Transformers	1	1010	May 26, 2021
`Trainer.predict` takes twice as long as progress bar shows 🤗Transformers	1	1198	June 7, 2021
Trainer class, compute_metrics and EvalPrediction 🤗Transformers	6	14633	October 28, 2020