Hugging Face Forums

Error loading finetuned llama2 model while running inference

Amazon SageMaker

Mit1208 September 20, 2023, 5:32pm 28

try with ml.g5.12xlarge according to AWS guide.

Topic		Replies	Views	Activity
ValueError: Could not load model /opt/ml/model with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.llama.modeling_llama.LlamaForCausalLM'>) Amazon SageMaker	0	394	March 13, 2024
QLoRA trained LLaMA2 13B deployment error on Sagemaker using text generation inference image Amazon SageMaker	14	2988	August 18, 2023
Error hosting endpoint when deploying model Amazon SageMaker	2	3065	March 27, 2024
Inference failed for FLAN-UL2(20B) on SageMaker Amazon SageMaker	6	2167	April 4, 2023
Deploying Fine-Tune Falcon 40B with QLoRA on Sagemaker Inference Error Amazon SageMaker	29	6842	January 8, 2024