Hi everyone, I would like to optimize AND quantize a NER model I have fine-tuned. I see that with the Optimum library it is easy to do both things separately, but I haven’t still managed to optimize and quantize the same model. Back with ONNXRuntime it was possible and easy to do. How can this be d…

@jorgealro , that is already possible and supported you can provide the file_name when either loading a ORTModel or creating a Optimizer/Quantizer. This is explained and document in our documentation.

Optimize AND quantize with Optimum

🤗Optimum

jorgealro September 29, 2022, 3:50pm 7

Yes, that works for me

Thank you very much!!

1 Like

Topic		Replies	Views
Optimum library optimization and quantization fails 🤗Optimum	8	1605	February 22, 2025
Optimisation and Quantization of Tensorflow Model 🤗Optimum	1	664	May 3, 2023
Getting ValueError when exporting model to ONNX using optimum 🤗Optimum	16	5100	November 25, 2022
Optimum & RoBERTa: how far can we trust a quantized model against its pytorch version? 🤗Optimum	10	2421	July 27, 2022
Quantized Model size difference when using Optimum vs. Onnxruntime 🤗Optimum	3	1536	July 14, 2022

Optimize AND quantize with Optimum

Related topics