Hey guys, I’ve managed to create a quantize version of gpt2 using onnxruntime but i don’t seem to be able to run it for some reason. anyone has a tutorial for it? also how does the “generate” method of the model will work with that any ideas?

Gpt2 inference with onnx and quantize

yanagar25 February 3, 2021, 11:29am 7

I will definitely look into it! thank you again

1 Like