Hey guys, I’ve managed to create a quantize version of gpt2 using onnxruntime but i don’t seem to be able to run it for some reason. anyone has a tutorial for it? also how does the “generate” method of the model will work with that any ideas?

Gpt2 inference with onnx and quantize

yanagar25 February 3, 2021, 10:43am 5

thank you so much for the reply!

1 Like