I’m trying to export the torch model into ONNX format using optimum and this warning came out:
2023-11-09 23:05:31.461232604 [W:onnxruntime:, session_state.cc:1162 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2023-11-09 23:05:31.461260995 [W:onnxruntime:, session_state.cc:1164 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
Code to reproduce:
from optimum.onnxruntime import ORTModelForFeatureExtraction from transformers import AutoTokenizer from pathlib import Path model_id="intfloat/e5-large-v2" onnx_path = Path("onnx") # load vanilla transformers and convert to onnx onnx_model = ORTModelForFeatureExtraction.from_pretrained(model_id, export=True, provider="CUDAExecutionProvider") tokenizer = AutoTokenizer.from_pretrained(model_id)