I’m trying to deploy my finetuned LLama3 model on Aws, so the first step to create an endpoint I used instance_type=“ml.g5.4xlarge”, This is my code: import json import sagemaker import boto3 from sagemaker.huggingface import HuggingFaceModel, get_huggingface_llm_image_uri try: role = sagema…

Error deploying endpoint on Aws

philschmid August 23, 2024, 11:52am 5

Try TGI 2.2.0

Topic		Replies	Views
Error hosting endpoint when deploying model in sagemaker Models	0	90	July 20, 2024
Error hosting endpoint when deploying model Amazon SageMaker	2	3051	March 27, 2024
Deploying TheBloke/Luna-AI-Llama2-Uncensored-GGML Amazon SageMaker	0	844	September 11, 2023
Need help deploying a HF model to AWS Sagemaker Amazon SageMaker	3	165	September 27, 2024
Error loading finetuned llama2 model while running inference Amazon SageMaker	27	4809	September 20, 2023