When trying to use Huggingface estimator on sagemaker, Run training on Amazon SageMaker e.g. # create the Estimator huggingface_estimator = HuggingFace( entry_point='train.py', source_dir='./scripts', instance_type='ml.p3.2xlarge', instance_count=1, role=role…

I published a repo that shows how you can extend the existing HF DLCs in order to use transformers versions higher than 4.17.0. The repo shows how to extend the Inference Container, but the same applies to the Training DLC. Hope that helps! [image] GitHub - marshmellow77/sm-extend-contain…

Huggingface / Pytorch versions on Sagemaker

Amazon SageMaker

marshmellow77 November 18, 2022, 6:24pm 4

Yup, that would be in this post

Topic		Replies	Views
Highest Transformer vers. that works with Sagemaker? Amazon SageMaker	2	459	September 1, 2021
Training on Sagemaker with Trainer() Instance Amazon SageMaker	6	2269	November 3, 2021
How is transfomers_verion and pytorch_version determined Amazon SageMaker	1	2858	November 9, 2022
Transformers 4.6.0 on SageMaker? Amazon SageMaker	14	4509	September 9, 2022
HuggingFace with Sagemaker tutorial doesn't work Amazon SageMaker	5	1243	August 25, 2021

Huggingface / Pytorch versions on Sagemaker

Related topics