I want to export a pytorch model into onnx format. Here is model:
from transformers import AutoProcessor, PaliGemmaForConditionalGeneration, BitsAndBytesConfig
model_id = "google/paligemma-3b-mix-224"
quantization_config = BitsAndBytesConfig(load_in_8bit=True)
model = PaliGemmaForConditionalGeneration.from_pretrained(
model_id, quantization_config=quantization_config
).eval()
processor = AutoProcessor.from_pretrained(model_id)
And my exporting onnx code is:
torch.onnx.export(
model,
(input_ids, attention_mask, pixel_values),
"model.onnx",
input_names=['input_ids', 'attention_mask', 'pixel_values'],
output_names=['output']
)
The above code throws this error:
/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py in new_forward(module, *args, **kwargs)
164 output = module._old_forward(*args, **kwargs)
165 else:
--> 166 output = module._old_forward(*args, **kwargs)
167 return module._hf_hook.post_forward(module, output)
168
/usr/local/lib/python3.10/dist-packages/transformers/models/siglip/modeling_siglip.py in forward(self, pixel_values, interpolate_pos_encoding)
302
303 def forward(self, pixel_values: torch.FloatTensor, interpolate_pos_encoding=False) -> torch.Tensor:
--> 304 _, _, height, width = pixel_values.shape
305 patch_embeds = self.patch_embedding(pixel_values) # shape = [*, width, grid, grid]
306 embeddings = patch_embeds.flatten(2).transpose(1, 2)
ValueError: not enough values to unpack (expected 4, got 2)
I have tried couple of solution, I think this is because when passing input to onnx, it equivalent to (This code throw the same above error):
model(input_ids, attention_mask, pixel_values)
However, the correct way to pass input to model is using kwargs
model(input_ids=input_ids, attention_mask=attention_mask, pixel_values=pixel_values)
How can I passed the keywords into onnx exporter, or is there any better approarch to export onnx format. Please help me thanks