Hi @valhalla @patrickvonplaten , I was working with onnx_transformers and using onnx for GPT-2 model and text-generation task. I used transformer pipeline for text-generation and the runtime for generating text was a bit high(20~30s) and I’ve tried using different approaches like using cronjobs to h…

Using onnx for text-generation with GPT-2

nielsr July 25, 2022, 11:35am 4

1 Like

Topic		Replies	Views
Accelerated gpt2-chinese-cluecorpussmall model Beginners	0	411	September 17, 2021
Gpt2 inference with onnx and quantize Beginners	6	3861	February 3, 2021
Speech2Text transformer to ONNX conversion 🤗Transformers	0	692	June 20, 2021
:rocket: Optimum Transformers: accelerated NLP pipelines with Infinity speed 🤗Transformers	4	665	March 25, 2022
Optimize large scale transformer model inference with ONNX Runtime Models	0	381	January 18, 2022