When I try to deploy a HuggingFace Sagemaker model with elastic inference (denoted by the accelerator_type parameter) I get an error. Deploy Snippet: predictor = huggingface_model.deploy( initial_instance_count=1, instance_type="ml.t2.medium", accelerator_type='ml.eia2.medium' ) Error…

Deploying Huggingface Sagemaker Models with Elastic Inference

philschmid July 28, 2021, 6:46am 2

Sadly speaking we don’t have EI DLCs yet. We are working on it and it is on the roadmap with one of the highest priorities.
I would update this thread here when I got any news.

4 Likes

Topic		Replies	Views
Inference failed for FLAN-UL2(20B) on SageMaker Amazon SageMaker	6	2166	April 4, 2023
About the Amazon SageMaker category Amazon SageMaker	25	4102	August 5, 2021
Deploying TheBloke/Luna-AI-Llama2-Uncensored-GGML Amazon SageMaker	0	844	September 11, 2023
Emotion Model: Additional inference parameter not processed in Sagemaker inferentia instance Amazon SageMaker	1	278	July 17, 2023
How do I deploy a hub model to SageMaker and give it a GPU (not Elastic Inference)? Amazon SageMaker	4	3380	February 15, 2022

Deploying Huggingface Sagemaker Models with Elastic Inference

Related topics