Ctransformers error : Failed to create LLM 'stablelm'

🀗Transformers
1

I often get error from ctransformers when importing some GGUF models with model_type = stablelm in Google Colab. Here is the code example

from ctransformers import AutoModelForCausalLM

llm = AutoModelForCausalLM.from_pretrained("TheBloke/Marx-3B-v3-GGUF", model_file="marx-3b-v3.Q3_K_S.gguf", model_type='stablelm')
print(llm("AI is going to"))

It actually downloaded the .gguf file from repo, but returned this error message

RuntimeError: Failed to create LLM 'stablelm' from '/root/.cache/huggingface/hub/models--TheBloke--Marx-3B-v3-GGUF/blobs/fb16032da1b4f68d465cb7b2164c5305be8a008657ed0cd6cbb91b3a94b032ee'.
1 Like
2

Hi @jonathanjordan21, please raise this issue on ctransformers library.