Deploying open llm - google/flan-t5-large model on AWS inferentia2

divyaswamy87 · September 14, 2023, 5:09pm

Hi,
I am trying to deploy open llm - google/flan-t5-large model on AWS inferentia2. Can anyone help me with resources on how to deploy LLM on Inferentia 2. I have seen some doc on Llambda nothing on FLAN-T5

Thank you

Topic		Replies	Views
🤗 LLM Inference Container for SageMaker Beginners	1	199	June 7, 2023
Deploying T5-style models via Sagemaker Endpoint: 'T5LayerFF' object has no attribute 'config' Amazon SageMaker	5	1354	November 7, 2022
ModelError when deploying google/flan-t5-xl Amazon SageMaker	1	374	July 31, 2023
Inference failed for FLAN-UL2(20B) on SageMaker Amazon SageMaker	6	1743	April 4, 2023
Is llama2 supported by the Hugging Face Text Generation Inference (TGI) Deep Learning Container on Amazon SageMaker? 🤗Transformers	0	469	August 3, 2023

Deploying open llm - google/flan-t5-large model on AWS inferentia2

Related Topics