ValueError: Model requires 4 inputs. Input Feed contains 2 ONNX

i try to export T5 model to ONNX but when a try to predict something it show a error:
sess = ort.InferenceSession(‘onnx\model.onnx’)
inputs = tokenizer(“something in here”, return_tensors=“np”)
outputs = sess.run( None,inputs )

ValueError: Model requires 4 inputs. Input Feed contains 2

Maybe @lewtun will have an idea?

Hey @Slcreator the number of inputs you need for inference depends on the type of head you chose to export the model with (i.e. the feature arg in the transformers.onnx package).

For example, here’s how you can figure out which inputs are needed for say the default feature of a T5 checkpoint:

from transformers.models.t5 import T5Config, T5OnnxConfig

config = T5Config.from_pretrained("t5-small")
onnx_config = T5OnnxConfig(config=config, task="default")
# returns
# {'input_ids': {0: 'batch', 1: 'encoder_sequence'},
#  'attention_mask': {0: 'batch', 1: 'encoder_sequence'},
#  'decoder_input_ids': {0: 'batch', 1: 'decoder_sequence'},
#  'decoder_attention_mask': {0: 'batch', 1: 'decoder_sequence'}}
onnx_config.inputs

You can change the task arg to match the type of head you’re exporting and that will tell you what the model expects as inputs.

Btw we’ve recently added a pipeline() function to the optimum library, which makes it quite simple to run inference on ONNX Runtime. There’s a guide here if you’re interested: Optimum pipelines for inference

1 Like

thanks very much, i’m going to check it again

1 Like

Hello, I am trying to use Bigbirdpegasus and getting the same error. Model requires 4 inputs but how can we provide decoder attention and mask beforehand? isn’t that supposed to be generated after we get output from the model?
Can you please share the code to export bigbird for summarization?
Here is the code I am using:

from pathlib import Path
from transformers.onnx import export
from transformers import AutoTokenizer, AutoModel

onnx_path = Path(“model.onnx”)
model_ckpt = “google/bigbird-pegasus-large-pubmed”
base_model = AutoModel.from_pretrained(model_ckpt)
tokenizer = AutoTokenizer.from_pretrained(model_ckpt)
onnx_inputs, onnx_outputs = export(tokenizer, base_model, onnx_config, onnx_config.default_onnx_opset, onnx_path)
tokenizer = AutoTokenizer.from_pretrained(“google/bigbird-pegasus-large-pubmed”)
session = InferenceSession(“model.onnx”)
inputs = tokenizer(“Summarize this text”, return_tensors=“np”)
inputs = {k: v.astype(np.int64) for k, v in inputs.items()}
outputs = session.run(output_names=[“last_hidden_state”], input_feed=inputs)