Hi, I was experimenting with many models include GPT2, T5 etc. But it seems like Tensorflow models are too slow for same type of generation comparing to Tensorflow, whether it is greedy, beam etc . Any specific reasons for this? Thanks

Why Tensorflow Models are way slower than Pytorch models, for autoregressive modeling?

clem October 8, 2020, 7:08am 6

Maybe @jplu has some insights!

1 Like

Topic		Replies	Views
Tensorflow Models are way slower than Pytorch models, for autoregressive generation? 🤗Transformers	3	389	July 26, 2022
What makes the built-in generate method faster than a crude manual implementation? 🤗Transformers	3	1981	January 19, 2024
Is model.generate slower than model forward call? 🤗Transformers	1	180	August 18, 2024
BERT model is slow in Pytorch 🤗Transformers	5	633	November 30, 2023
Advice to speed and performance 🤗Transformers	4	7226	December 7, 2020