Huggingface_hub integration: ModuleNotFoundError: No module named 'huggingface_hub'

Amazon SageMaker
1

Hi Philipp,

I have been trying to use the new functionally of push to hub on my script and I could not even past the installation, I ran the: !pip install “sagemaker==2.69.0” “transformers==4.12.3” --upgrade command and for some reason sagemaker is not getting updated.

I am using a notebook instance.

Thanks,

Jorge

2

Hey @Jorgeutd,

The sagemaker notebooks are currently containing an incompatible/old version of boto3 with the latest dataset and transformers.

Thats how I solved it:

!pip install "sagemaker>=2.69.0" "transformers==4.12.3" --upgrade
# using older dataset due to incompatibility of sagemaker notebook & aws-cli with > s3fs and fsspec to >= 2021.10
!pip install  "datasets==1.13" --upgrade

BTW. we also have an example notebook on how to push models to the hub during sagemaker training.

1 Like
3

Thank you @philschmid . For some reason is not working for me.

Yes, that was my goal, to push the model to the hub during sagemaker training.

I will try the old way.

Thanks,

Jorge

4

What error are you seeing? Which kernel do you use?

5

Sagemaker notebook instace: Kernel: Conda_pytorch_latest_p37

error: ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: ‘/home/ec2-user/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/PyYAML-6.0.dist-info/METADATA’

Script: !pip install “sagemaker>=2.69.0” “transformers==4.12.3” --upgrade

using older dataset due to incompatibility of sagemaker notebook & aws-cli with > s3fs and fsspec to >= 2021.10

!pip install “datasets==1.13” --upgrade

6

could you try conda_pytorch_p36 kernel?

7

Thank you @philschmid let me do that now.

Any recommendations on how start with NER tasks?

Thanks,

Jorge