Hi HuggingFace community, I’m attempting to deploy a fine-tuned T5 model for summarization using a SageMaker Endpoint. The endpoint is deployed successfully with the following code: from sagemaker.huggingface.model import HuggingFaceModel huggingface_model = HuggingFaceModel( model_data="s3://…

Thanks @philschmid for your comment, but that wasn’t the problem in this case, the model.tar.gz was fine. I just found the solution - I needed to add an extra parameter to the model like so: env = {'HF_TASK': 'summarization'} huggingface_model = HuggingFaceModel( model_data="s3://my-s3-path/mo…

Calling Sagemaker Endpoint for fine-tuned summarization model

Amazon SageMaker

philschmid February 22, 2023, 8:43am 9

Hello @Pidem,

when you implement a custom inference.py you don’t have to pass any ENV.

Topic		Replies	Views
Deploying T5-style models via Sagemaker Endpoint: 'T5LayerFF' object has no attribute 'config' Amazon SageMaker	5	1484	November 7, 2022
InternalServerException from bart model created from s3 Amazon SageMaker	1	403	May 22, 2023
Inference failed for FLAN-UL2(20B) on SageMaker Amazon SageMaker	6	2207	April 4, 2023
Sagemaker Text Summarization Fine Tuning Job failing Amazon SageMaker	6	1610	June 9, 2022
How to Create Model in SageMaker Console from .tar.gz Amazon SageMaker	7	10469	March 10, 2022

Calling Sagemaker Endpoint for fine-tuned summarization model

Related topics