I’m working with the phi-4 MM model that I’ve fine-tuned for a vision task, and I’m trying to: Quantize the model (preferably using INT8 or 8-bit methods like 'AWQ , Autoround orbitsandbytes ). Convert the model to ONNX format for deployment. So far, I haven’t found clear documentation for quant…

Help with Quantizing phi-4 MM Fine-Tuned Vision Model and Converting to ONNX

Meemank May 2, 2025, 9:51am 4

Do we have any sample notebook where we are quantizing phi4-mm-instruct finetuned vision model?

1 Like

Topic		Replies	Views
🔧 Optimizing Phi-4 MM Instruct Vision Model (ONNX Inference) Intermediate	1	43	April 24, 2025
Quantization of facebook/opt-13b model 🤗Transformers	0	997	July 28, 2022
SmolVLM 8bit Quantization Problem Models	3	442	November 29, 2024
ONNX conversion 🤗Transformers	0	284	July 8, 2021
Improving Quantization Accuracy for ONNX Models with Optimum 🤗Optimum	0	713	February 8, 2024