When trying to use Huggingface estimator on sagemaker, Run training on Amazon SageMaker e.g. # create the Estimator huggingface_estimator = HuggingFace( entry_point='train.py', source_dir='./scripts', instance_type='ml.p3.2xlarge', instance_count=1, role=role…

I published a repo that shows how you can extend the existing HF DLCs in order to use transformers versions higher than 4.17.0. The repo shows how to extend the Inference Container, but the same applies to the Training DLC. Hope that helps! [image] GitHub - marshmellow77/sm-extend-contain…

Huggingface / Pytorch versions on Sagemaker

Amazon SageMaker

marshmellow77 November 18, 2022, 3:25pm 2

Hi @alvations - this thread should hopefully answer all your questions

1 Like

Topic		Replies	Views
Transformers 4.6.0 on SageMaker? Amazon SageMaker	14	3863	September 9, 2022
HuggingFace with Sagemaker tutorial doesn't work Amazon SageMaker	5	913	August 25, 2021
Huggingface Training Containers Beginners	0	71	March 15, 2024
Package errors running huggingface estimator on sagemaker Beginners	1	732	February 9, 2023
Sagemaker Huggingface Transformers support: New updates? Amazon SageMaker	1	1402	November 3, 2022

Huggingface / Pytorch versions on Sagemaker

Related Topics