Hey guys, I’m trying to load this GGUF model dolphin-2.0-mistral-7b.Q4_K_M.gguf with this Python code: model_name = "TheBloke/dolphin-2.0-mistral-7B-GGUF" model_file = "dolphin-2.0-mistral-7b.Q4_K_M.gguf" tokenizer = AutoTokenizer.from_pretrained(model_name, gguf_file=model_file) model = AutoModel…

ValueError: cannot reshape array of size (GGUF)

jpuser1 July 31, 2024, 9:51pm 4

@H4rryM3ss I was able to get past this error by cloning the huggingface/transformers repository and running the latest code from there. At least the model loads now.

Topic		Replies	Views
Unable to run gguf model Models	1	950	January 6, 2025
NotImplementedError: ggml_type 21 not implemented 🤗Transformers	2	82	September 23, 2024
Failed to create LLM 'llama' from .GGUF Beginners	0	298	December 25, 2024
Why I am getting this problem while running any of the GGUF model instead of with .bin model Models	1	2622	November 7, 2023
Hugging Face to GGUF Conversion Broken? 🤗Hub	1	5327	February 11, 2024

ValueError: cannot reshape array of size (GGUF)

Related topics