Hi, I just watched the video of the Workshop: Going Production: Deploying, Scaling & Monitoring Hugging Face Transformer models (11/02/2021) from Hugging Face. With the informations about how to deploy ( timeline start: 28:14 ), I created a notebook instance (type: ml.m5.xlarge) on AWS SageMaker whe…

How to deploy a T5 model to AWS SageMaker for fast inference?

GenV February 28, 2022, 2:38pm 13

Hi @pierreguillou, I have the same problem. I created a post here. Did you solve the problem?

Topic		Replies	Views
Deploying open llm - google/flan-t5-large model on AWS inferentia2 Amazon SageMaker	0	441	September 14, 2023
Deploying T5-style models via Sagemaker Endpoint: 'T5LayerFF' object has no attribute 'config' Amazon SageMaker	5	1464	November 7, 2022
Help for inference.py code Amazon SageMaker	10	4001	March 8, 2022
Inference Hyperparameters Amazon SageMaker	29	4835	October 8, 2021
Deploying Open AI's whisper on Sagemaker Amazon SageMaker	54	16194	April 12, 2024