I used optimum and onnxruntime to optimize and quantize a roberta squad QA model using the example from this blog huggingface/blog/blob/main/optimum-inference.md --- title: 'Accelerated Inference with Optimum and Transformers Pipelines' thumbnail: /blog/assets/66_optimum_inference/t…

Optimum library optimization and quantization fails

ddahlmeier February 10, 2024, 9:05am 2

Quantizing from the non-optimized model works

2 Likes

Optimize AND quantize with Optimum