Hi @echarlaix,
I’m using a notebook in AWS Sagemaker and I wanted to test Optimum by running the code from Optimum Inference with ONNX Runtime.
It works without problem in Google Colab but not in an AWS Sagemaker notebook: in an AWS Sagemaker notebook, I can not import a class ORTModelFor(…).
Do you know why?
!python -m pip install optimum[onnxruntime]
from optimum.onnxruntime import ORTModelForQuestionAnswering
It looks like that optimum[onnxruntime] is not installed correctly.
Could you please print the versions you have installed in sagemaker for optimum and onnxruntime?
Hi. When I run !python -m pip install optimum[onnxruntime] in a notebook of AWS Sagemaker, I get the following information:
Requirement already satisfied: optimum[onnxruntime] in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (1.1.1)
Requirement already satisfied: transformers>=4.15.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from optimum[onnxruntime]) (4.18.0)
Requirement already satisfied: coloredlogs in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from optimum[onnxruntime]) (15.0.1)
Requirement already satisfied: sympy in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from optimum[onnxruntime]) (1.7.1)
Requirement already satisfied: packaging in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from optimum[onnxruntime]) (21.3)
Requirement already satisfied: torch>=1.9 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from optimum[onnxruntime]) (1.10.2)
Requirement already satisfied: onnx in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from optimum[onnxruntime]) (1.10.0)
Requirement already satisfied: onnxruntime in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from optimum[onnxruntime]) (1.10.0)
Requirement already satisfied: datasets>=1.2.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from optimum[onnxruntime]) (2.3.2)
Requirement already satisfied: multiprocess in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (0.70.12.2)
Requirement already satisfied: numpy>=1.17 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (1.19.2)
Requirement already satisfied: fsspec[http]>=2021.05.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (2022.1.0)
Requirement already satisfied: responses<0.19 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (0.17.0)
Requirement already satisfied: requests>=2.19.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (2.26.0)
Requirement already satisfied: xxhash in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (3.0.0)
Requirement already satisfied: dataclasses in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (0.8)
Requirement already satisfied: huggingface-hub<1.0.0,>=0.1.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (0.4.0)
Requirement already satisfied: pyarrow>=6.0.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (6.0.1)
Requirement already satisfied: tqdm>=4.62.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (4.62.3)
Requirement already satisfied: importlib-metadata in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (3.7.0)
Requirement already satisfied: dill<0.3.6 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (0.3.4)
Requirement already satisfied: aiohttp in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (3.8.1)
Requirement already satisfied: pandas in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from datasets>=1.2.1->optimum[onnxruntime]) (1.1.5)
Requirement already satisfied: typing-extensions in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from torch>=1.9->optimum[onnxruntime]) (4.0.1)
Requirement already satisfied: regex!=2019.12.17 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from transformers>=4.15.0->optimum[onnxruntime]) (2020.11.13)
Requirement already satisfied: filelock in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from transformers>=4.15.0->optimum[onnxruntime]) (3.4.1)
Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from transformers>=4.15.0->optimum[onnxruntime]) (0.12.1)
Requirement already satisfied: pyyaml>=5.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from transformers>=4.15.0->optimum[onnxruntime]) (5.4.1)
Requirement already satisfied: sacremoses in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from transformers>=4.15.0->optimum[onnxruntime]) (0.0.53)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from packaging->optimum[onnxruntime]) (2.4.7)
Requirement already satisfied: humanfriendly>=9.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from coloredlogs->optimum[onnxruntime]) (10.0)
Requirement already satisfied: six in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from onnx->optimum[onnxruntime]) (1.15.0)
Requirement already satisfied: protobuf in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from onnx->optimum[onnxruntime]) (3.19.1)
Requirement already satisfied: flatbuffers in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from onnxruntime->optimum[onnxruntime]) (2.0)
Requirement already satisfied: mpmath>=0.19 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sympy->optimum[onnxruntime]) (1.2.1)
Requirement already satisfied: idna<4,>=2.5 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from requests>=2.19.0->datasets>=1.2.1->optimum[onnxruntime]) (3.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from requests>=2.19.0->datasets>=1.2.1->optimum[onnxruntime]) (1.26.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from requests>=2.19.0->datasets>=1.2.1->optimum[onnxruntime]) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from requests>=2.19.0->datasets>=1.2.1->optimum[onnxruntime]) (2021.5.30)
Requirement already satisfied: attrs>=17.3.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from aiohttp->datasets>=1.2.1->optimum[onnxruntime]) (20.3.0)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from aiohttp->datasets>=1.2.1->optimum[onnxruntime]) (4.0.1)
Requirement already satisfied: idna-ssl>=1.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from aiohttp->datasets>=1.2.1->optimum[onnxruntime]) (1.1.0)
Requirement already satisfied: frozenlist>=1.1.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from aiohttp->datasets>=1.2.1->optimum[onnxruntime]) (1.2.0)
Requirement already satisfied: aiosignal>=1.1.2 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from aiohttp->datasets>=1.2.1->optimum[onnxruntime]) (1.2.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from aiohttp->datasets>=1.2.1->optimum[onnxruntime]) (1.6.3)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from aiohttp->datasets>=1.2.1->optimum[onnxruntime]) (5.1.0)
Requirement already satisfied: asynctest==0.13.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from aiohttp->datasets>=1.2.1->optimum[onnxruntime]) (0.13.0)
Requirement already satisfied: zipp>=0.5 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from importlib-metadata->datasets>=1.2.1->optimum[onnxruntime]) (3.4.0)
Requirement already satisfied: python-dateutil>=2.7.3 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from pandas->datasets>=1.2.1->optimum[onnxruntime]) (2.8.1)
Requirement already satisfied: pytz>=2017.2 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from pandas->datasets>=1.2.1->optimum[onnxruntime]) (2021.1)
Requirement already satisfied: click in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sacremoses->transformers>=4.15.0->optimum[onnxruntime]) (7.1.2)
Requirement already satisfied: joblib in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sacremoses->transformers>=4.15.0->optimum[onnxruntime]) (1.0.1)
Hello Pierre, ORTModelFor... were added in Optimum 1.2.0 and you are running Optimum 1.1.1. I believe the reason python -m pip install optimum[onnxruntime] did not install the latest Optimum version is because you use Python 3.6, which is not supported from Optimum 1.2.0 onwards.
Could you try to upgrade to Python 3.7?