Error loading finetuned llama2 model while running inference

Harisvossos10 September 19, 2023, 1:12pm 21

I get the same error here for CodeLlama-7b-hf-Instruct.

I even included a requirements.txt file in model.tar.gz requiring a transformers==4.33.2 but it doesnt work.
Any ideas?

Topic		Replies	Views
ValueError: Could not load model /opt/ml/model with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.llama.modeling_llama.LlamaForCausalLM'>) Amazon SageMaker	0	405	March 13, 2024
QLoRA trained LLaMA2 13B deployment error on Sagemaker using text generation inference image Amazon SageMaker	14	3023	August 18, 2023
Error hosting endpoint when deploying model Amazon SageMaker	2	3108	March 27, 2024
Inference failed for FLAN-UL2(20B) on SageMaker Amazon SageMaker	6	2207	April 4, 2023
Deploying Fine-Tune Falcon 40B with QLoRA on Sagemaker Inference Error Amazon SageMaker	29	6935	January 8, 2024