Calling Sagemaker Endpoint for fine-tuned summarization model

Amazon SageMaker
3

Thanks @philschmid for your comment, but that wasn’t the problem in this case, the model.tar.gz was fine.

I just found the solution - I needed to add an extra parameter to the model like so:

env = {'HF_TASK': 'summarization'}

huggingface_model = HuggingFaceModel(
   model_data="s3://my-s3-path/model.tar.gz",
   role=role,
    env=env,
   transformers_version="4.6",
   pytorch_version="1.7",
   py_version='py36',
)

This is not in the documentation anywhere for fine-tuned models, but does appear in the tests for the inference package here: sagemaker-huggingface-inference-toolkit/test_models_from_hub.py at main · aws/sagemaker-huggingface-inference-toolkit · GitHub