ImportError: cannot import name 'send_example_telemetry' from 'transformers.utils' | upgrade the transformer version in AWS Sagemaker DLC

Amazon SageMaker
1

Hi HF mighty community,

I am getting following error when executing fine tuning NER code on Sagemaker.

ErrorMessage "ImportError: cannot import name ‘send_example_telemetry’ from ‘transformers.utils’ (/opt/conda/lib/python3.8/site-packages/transformers/utils/init.py)

I am able to fine tune the run_ner.py model on my local machine using latest version of Transformer library which is 4.21.0.dev0. However, when I try to run the model on Sagemaker, the training job is always failing throwing above Import error.
Sagemaker DLC code snippet:

huggingface_estimator = HuggingFace(
	entry_point='run_ner.py',
	source_dir='./token-classification',
	instance_type='ml.p3.2xlarge',
	instance_count=1,
	role=role,
	#git_config=git_config,
	transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	hyperparameters = hyperparameters
)

I tried to update the transformer parameter in HuggingFace() to
transformers_version=‘4.21.0’, but i get following Value error.

ValueError: Unsupported huggingface version: 4.21.0. You may need to upgrade your SDK version (pip install -U sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.4.2, 4.5.0, 4.6.1, 4.10.2, 4.11.0, 4.12.3, 4.17.0, 4.4, 4.5, 4.6, 4.10, 4.11, 4.12, 4.17

My hunch is, Sagemaker DLC is using older version of transformer, I am getting ImportError. Is there any way to upgrade the Sagemaker DLC transformer version or solve this issue in another way?

Thank you.
-Dinesh

2

How does your run_ner.py look? It looks like you are using a newer run_ner.py than the version in the container. You can get the correct example from the corresponding release tag, .e.g. for 4.17.0 it is : transformers/run_ner.py at v4.17.0 · huggingface/transformers · GitHub

1 Like
3

Thank you @philschmid for your swift response.
Good catch! I clone the main branch of ‘transformers’ from git. And in main branch run_ner.py has below additional code snippet

# Sending telemetry. Tracking the example usage helps us better allocate resources to maintain them. The
# information sent is the one passed as arguments along with your Python/PyTorch versions.
send_example_telemetry("run_ner", model_args, data_args)

Agree, switching the branch to 4.17.0 should solve this issue!

Also, few minutes ago, I found a way to upgrade the transformer version to 4.21.0.dev0 by adding one line of code in requirement.txt

git+https://github.com/huggingface/transformers

Thanks!

1 Like