ValueError: cannot reshape array of size (GGUF)

jpuser1 · July 31, 2024, 7:20pm

Possibly due to this bug:

ValueError: cannot reshape array of size <number> into shape

opened 03:01PM - 30 Jul 24 UTC

bug

### System Info Python 3.11.2 Debian 12 ctransformers 0.2.27 tran…sformers 4.42.4 ### Who can help? @ArthurZucker ### Information - [X] The official example scripts - [ ] My own modified scripts ### Tasks - [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...) - [ ] My own task or dataset (give details below) ### Reproduction Following the example on Hugging Face: https://huggingface.co/docs/transformers/en/gguf ``` import torch from transformers import AutoModelForCausalLM, AutoTokenizer from huggingface_hub import hf_hub_download from llama_cpp import Llama # Ensure the model is loaded on the GPU device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') print(f"device set to: {device}") # Load the model and tokenizer model_name = "TheBloke/dolphin-2.0-mistral-7B-GGUF" model_file = "dolphin-2.0-mistral-7b.Q4_K_M.gguf" print(f"Loading tokenizer from {model_name}.") tokenizer = AutoTokenizer.from_pretrained(model_name, gguf_file=model_file) print(f"Loading model from repo {model_name}. Model file name {model_file}.") model = AutoModelForCausalLM.from_pretrained(model_name, gguf_file=model_file) ``` I get ``` ValueError: cannot reshape array of size 36864000 into shape (222,72) ``` ### Expected behavior Model should be loaded.

Topic		Replies	Views
Unable to run gguf model Models	1	950	January 6, 2025
NotImplementedError: ggml_type 21 not implemented 🤗Transformers	2	82	September 23, 2024
Failed to create LLM 'llama' from .GGUF Beginners	0	298	December 25, 2024
Why I am getting this problem while running any of the GGUF model instead of with .bin model Models	1	2622	November 7, 2023
Hugging Face to GGUF Conversion Broken? 🤗Hub	1	5327	February 11, 2024

ValueError: cannot reshape array of size (GGUF)

Related topics