The download of a model is too slow

Beginners
1

Hi everyone! :wave:

I am trying to download the Falcon-7B model from its repo into my local machine and the download speed is very slow. I get an average download speed of 2.50 Mbps and it takes nearly 1h to download the 15GB weight files (I have good internet connection with at least 15Mbps)

When I try downloading the same model in Google Colab the download speed is way faster. The first time I tried to download the model it took less than 4 minutes, and the next ones ~10 minutes (I suppose Hugging Face servers slow down repetitive calls from the same user)

Here is the code I run to download the model (I don’t have GPU)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, AutoTokenizer

model_id = "tiiuae/falcon-7b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    device_map="auto", 
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    cache_dir="cache_models")

I’ve also tried cloning the repo through git lfs but after some minutes waiting I get a warning stating that the weight models could not be downloaded correctly. When I try to load the model from the cloned repo I get the error: “Unable to load weights from pytorch checkpoint file for ‘./cache_models/falcon-7b\pytorch_model-00001-of-00002.bin’ at ‘./cache_models/falcon-7b\pytorch_model-00001-of-00002.bin’. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.”

Appending the option from_tf=True yields the error: ‘list’ object has no attribute ‘endswith’ which I don’t know how to fix, but I suspect that the original error comes from the faulty download of the weights by git lfs

Does anyone have faced a similar problem or might know why is this happening? Is there any chance to speed up the download process?


PS. When I try to download other models like [MTP-7B](https://huggingface.co/mosaicml/mpt-7b) the download times remain the same. PC specifications:
  • i7-1065G7 CPU @ 1.30GHz
  • 12GB RAM
  • Windows 11
  • 200GB free disk space
3 Likes