HuggingFaceModel create fails with no GPU

Amazon SageMaker
1

I have a pretrained custom model with the model.tar.gz file located in S3. I want to create the HuggingFaceModel and the endpoint configuration in Sagemaker. My local machine (where I am running this) does not have a GPU, but the model will be deployed on a GPU instance in Sagemaker. When I try:

create Hugging Face Model Class

huggingface_model = HuggingFaceModel(
model_data=s3_location, # path to your model and script
role=role, # iam role with permissions to create an Endpoint
transformers_version=“4.37”, # transformers version used
pytorch_version=“2.1”, # pytorch version used
py_version=‘py310’, # python version used
name=‘cjk-model-1’,
}

I get an error this error,
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won’t be available and only tokenizers, configuration and file/data utilities can be used.

but I don’t understand why I should need pytorch since the model is already trained and it will run on Sagemaker?

1 Like
2

Transoformers library is essentially a wrapper that uses PyTorch or TensorFlow as its backend, so PyTorch is absolutely necessary.

For example, the model class inherits from torch.nn.Modules basically…

https://stackoverflow.com/questions/64337550/neither-pytorch-nor-tensorflow-2-0-have-been-found-models-wont-be-available

3

but Pytorch is used by the model/inference code, which is already wrapped into the model, and the huggingface image on sagemaker includes it, so why does the machine I’m deploying from require it?

1 Like
4

That’s true. What’s going on? For example, maybe SageMaker’s GPU has been updated and the old PyTorch is no longer compatible…?

pytorch_version=“2.4”, # pytorch version used