I have fine-tuned a model using QLoRA (4-bit) and then pushed it to the hub. Then, I tried to test it with inference API, but I get the following error: No package metadata was found for bitsandbytes The model: base model is meta-llama/Llama-2-7b-chat-hf fine-tuned on a custom dataset of 50 samp…

Inference API for fine-tuned model not working: No package metadata was found for bitsandbytes

FDeRubeis March 2, 2024, 4:16pm 3

Unfortunately not. Currently I just gave up But let me know if you find something.

1 Like