Tensorflow Models are way slower than Pytorch models, for autoregressive generation?

🀗Transformers
1

According to this post: Why Tensorflow Models are way slower than Pytorch models, for autoregressive modeling? , TF models are not optimized for NLG. Is it still the case? model.generate for TF models is upto 80x slower for me compared to PyTorch models.

2

cc @joaogante :slight_smile:

3

Hi @saurabh3949 :wave:

Your question comes with great timing! Tomorrow we will release a new version of transformers and one of its highlights is that TF NLG can now be massively accelerated, ever beyond PyTorch’s speed. There will be a blog post (also to be released tomorrow) explaining how, stay tuned!

1 Like
4

Thank you @joaogante ! This is very encouraging. You have put me in eager mode now :slight_smile: