Model won't load on custom inference endpoint

Inference Endpoints on the Hub
3

I managed to fix this issue by ensuring specific versions of TRL and peft were installed ! also the hf_hub did have a version that was not working.
here’s the requirements.txt for anyone that can be having the same issue in the future :

huggingface_hub
unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git
xformers
trl<0.9.0 
peft==0.11.1
bitsandbytes
transformers
1 Like