Using the currently latest version of Optimum, 1.18.0, I was able to convert sentence-transformers/all-MiniLM-L6-v2
into an ONNX model:
optimum-cli export onnx --task sentence-similarity --optimize O2 -m sentence-transformers/all-MiniLM-L6-v2 $(pwd)/models/all-MiniLM-L6-v2/onnx
When trying to use the resulting model with the currently latest version of Transformers.js, 2.16.1, I get this error message:
Error: Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model_load_utils.h:46 void onnxruntime::model_load_utils::ValidateOpsetForDomain(const std::unordered_map<std::basic_string<char>, int>&, const onnxruntime::logging::Logger&, bool, const string&, int) ONNX Runtime only *guarantees* support for models stamped with official released onnx opset versions. Opset 4 is under development and support for this is limited. The operator schemas and or other functionality may change before next ONNX release and in this case ONNX Runtime will not guarantee backward compatibility. Current official support for domain ai.onnx.ml is till opset 3
optimum-cli export onnx
supports an --opset
argument, but that seems to only control the opset for the ai.onnx
domain, not the ai.onnx.ml
domain. When I inspect my onnx model using Netron, I see that ai.onnx.ml
is indeed v4, which the error message complains is greater than the max. 3. Looking at onnx/docs/Versioning.md at main · onnx/onnx · GitHub, the last onnx
version to not support version 4 for ai.onnx.ml
was 1.14.1. I’m not sure if the versioning between onnx and onnxruntime-node is in sync, but Transformers.js uses onnxruntime-node 1.14, which is over a year old.
Basically, my conclusion is that Transformers.js lags far behind Optimum when it comes to the ONNX support and that they are therefore not compatible. It seems strange that that would be the case though, not least because one of the first things stated on Transformers.js is
“Transformers.js uses ONNX Runtime to run models in the browser. The best part about it, is that you can easily convert your pretrained PyTorch, TensorFlow, or JAX models to ONNX using Optimum.”
So maybe I’m missing something. How can I convert a model to ONNX and use it with Transformer.js?