It took me about 1 hour to load the model of llama2-7b-hf. It’s such weird. What can I do to resolve this issue? The code is attached as follows: from transformers import AutoModelForCausalLM model_dir = "meta-llama/Llama-2-7b-chat-hf" model = AutoModelForCausalLM.from_pretrained( model_di…

Why the model loading of llama2 is so slow?

saikumar305 April 24, 2024, 7:42am 7

@lepotatoguy , can you please provide bnb_config and device_map values in details?

Topic		Replies	Views
Loading a locally saved model is very slow 🤗Transformers	1	3969	July 10, 2024
Unable to load ALMA-13B model from HF 🤗Transformers	0	171	December 13, 2023
Finetuned llama7b model is 5x slower than hugingface raw model 🤗Transformers	2	1537	October 5, 2023
Load downloaded Llama2 model with Transformers Beginners	0	966	October 25, 2023
`text-generation` `Pipeline` prohibitively slow to load, even with cached model 🤗Transformers	1	4443	May 23, 2023