Can i export a VisionEncoderDecoder checkpoint to onnx

Can I export any hugging face checkpoint to onnx? If not, how do I go about it
@nielsr I want to export trocr checkpoint to onnx, is it possible.
I tried doing the same with a fine-tuned checkpoint of mine, it gives a KEY ERROR

KeyError: <class 'transformers.models.vision_encoder_decoder.configuration_vision_encoder_decoder.VisionEncoderDecoderConfig'>

When I do the same using “microsoft/trocr-large-printed” checkpoint, I get an exception saying

Do you know what might I be missing? Is it possible, if not , how do I go about it.

Pinging @lewtun here, to let him know people are also interested in exporting EncoderDecoder model classes to ONNX.

2 Likes

You can track the issue here @Kforcode

1 Like

Hey there! Any updates on this?

I’ve read through this documentation about implementing a custom ONNX configuration for unsupported architectures. The walk-through described here doesn’t look too difficult. But, I decided that before I attempted this I’d ask: is this possible with TrOCR in its current state, or is this irrelevant until the code is refactored (which I know you smart people are working on currently, and I thank you for it)?

We’re currently refactoring the internals of the ONNX exporter to handle multimodal models better here. Once that’s done, it should be relatively straightforward to export TrOCR and I’d be happy to guide you on that :slight_smile:

Oh wonderful! Thanks!

Hi there,
I am unable to export a fine-tuned TrOCR checkpoint as per the documentation for custom architectures. Is there a way to go about it?

I don’t know if anyone still needs it, but I cobbled together a rough script to convert some TrOCR models to ONNX.

Here it is. Hopefully it can help someone out.

Hi there @lewtun @nielsr , it would be cool if we can do the same for Donut model.

In addition, it would be cool to have also the posibility for optimum-intel.

I would be glad to help as it is Hacktoberfest :slight_smile:

VisionEncoderDecoderModel now has ONNX support, thanks to this PR: https://github.com/huggingface/transformers/pull/19254.

1 Like