Deploying Fine-Tune Falcon 40B with QLoRA on Sagemaker Inference Error

malterei July 21, 2023, 6:08pm 21

Missing links:

Topic		Replies	Views
QLoRA trained LLaMA2 13B deployment error on Sagemaker using text generation inference image Amazon SageMaker	14	3023	August 18, 2023
Unable to deploy Falcon 40b OASST1 model into SageMaker TGI container Amazon SageMaker	0	440	July 29, 2023
Falcon 40B instruct training with QLora, Sagemaker model artifact location Amazon SageMaker	3	410	September 21, 2023
Error loading finetuned llama2 model while running inference Amazon SageMaker	27	4874	September 20, 2023
Mistral AI Sagemaker deployment failing Amazon SageMaker	3	2091	December 29, 2023