Convert bert tokenizer to onnx

Beginners
#1

I was referring to the following blog to convert bert model to onnx.

here, to take the inference of bert tokenizer, I’ll have to pass the 2d arrays.

Is there a way, where I’ll be able to pass sentence as input to the onnx tokenizer and get encodings as output, so that I’ll be able to use the model platform-independent

#2

hi @hasak,

the tokenizer is independent of onnx / onnxruntime, so you could create a simple function that converts your string inputs into the numpy format that the onnxruntime session expects:

tokenizer = ...

def prepare_for_session(input: str):
    tokens = tokenizer(input, return_tensors="pt") 
    return {k: v.cpu().detach().numpy() for k, v in tokens.items()}

does this answer your question?

#3

I think using a separate tokenizer function does work, but bundling the tokenizer into the onnx object has it’s own benefits. For example, if someone wanted to take this model and deploy it client-side in the browser using ONNX.js.

There is currently preliminary support for Node bindings for the Tokenizers library, but that doesn’t support the latest LTS versions of node and won’t without a big overhaul of it’s “neon” dependency.

#4

Interesting suggestion @aitordiaz . The node bindings you are mentioning are tokenizers/README.md at main · huggingface/tokenizers · GitHub ?

I agree bundling a tokenizer in a format portable-friendly could be great not only for client-side in-browser inference, but other use cases as well (if we don’t want to use python for example). It seems like huge work though.