Hugging Face Forums

Error loading finetuned llama2 model while running inference

Amazon SageMaker

philschmid September 15, 2023, 12:05pm 12

Could you share how your model.tar.gz looks?

Topic		Replies	Views	Activity
ValueError: Could not load model /opt/ml/model with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.llama.modeling_llama.LlamaForCausalLM'>) Amazon SageMaker	0	405	March 13, 2024
QLoRA trained LLaMA2 13B deployment error on Sagemaker using text generation inference image Amazon SageMaker	14	3023	August 18, 2023
Error hosting endpoint when deploying model Amazon SageMaker	2	3108	March 27, 2024
Inference failed for FLAN-UL2(20B) on SageMaker Amazon SageMaker	6	2207	April 4, 2023
Deploying Fine-Tune Falcon 40B with QLoRA on Sagemaker Inference Error Amazon SageMaker	29	6935	January 8, 2024