Hi, for inference I created a deployment on AWS which is working fine when used for summarization It works fine on some texts, but as soon as I use a particular long text I get an error and I am not able to run any summarization taks again, even the ones with short input texts are no longer workin…

Sagemaker HuggingFaceModel crashed with CUDA error

philschmid May 17, 2023, 12:35pm 2

Topic		Replies	Views
Hugging Face BART Text Summarization Model-Sage Maker deployed after a month giving me an 400 error Models	1	583	May 2, 2023
CUDA error for inference on GPU instance Amazon SageMaker	2	766	May 16, 2023
Inference failed for FLAN-UL2(20B) on SageMaker Amazon SageMaker	6	2167	April 4, 2023
Calling Sagemaker Endpoint for fine-tuned summarization model Amazon SageMaker	15	5085	March 22, 2024
Inference error for FLAN-UL2 on AWS SageMaker Amazon SageMaker	1	957	April 3, 2023