Strange problem with huggingface-cli

jacek2024 · June 20, 2025, 11:46am

since yesterday I have some strange issues with huggingface-cli
usually when I download the file I can stop and resume download at any time
today my downloads stop around 95% and when I restart it, they start from 0%
it happened on two files, I tried clearing .cache 3 times already
any hints?

John6666 · June 20, 2025, 1:08pm

I think Xet is currently being adjusted. It might be related to that…

github.com/huggingface/xet-core

How can I have snapshot_download to have continue feature? Errors became very common

opened 10:30PM - 28 May 25 UTC

FurkanGozukara

enhancement

Whenever some error happens and i run same code, it starts from 0 It is XET ena…bled repo and hf xet installed I really need to have resume feature my entire code ``` from huggingface_hub import snapshot_download import os import argparse def download_models(target_dir=None): """ Download models from HuggingFace hub to specified directory Args: target_dir (str, optional): Target directory for downloads. If None, uses current working directory """ # Set repo ID repo_id = "MonsterMMORPG/Kohya_Train" # Use provided target dir or default to current working directory download_dir = target_dir if target_dir else os.getcwd() # Create target directory if it doesn't exist os.makedirs(download_dir, exist_ok=True) try: snapshot_download( local_dir=download_dir, repo_id=repo_id ) print(f"\nDOWNLOAD COMPLETED to: {download_dir}") print("Check folder content for downloaded files") except Exception as e: print(f"Error occurred during download: {str(e)}") if __name__ == "__main__": parser = argparse.ArgumentParser(description='Download models from HuggingFace hub') parser.add_argument('--dir', type=str, help='Target directory for downloads', default=None) args = parser.parse_args() download_models(args.dir) ```

jacek2024 · June 20, 2025, 2:21pm

thank you, so the problem is not on my side

John6666 · June 20, 2025, 2:28pm

Yeah, probably. If you updated the library beforehand, there may be bugs caused by the library upgrade, but if not, I think it’s either a server issue or, in rare cases, a network problem.

jacek2024 · June 20, 2025, 2:35pm

I tried pip install --upgrade huggingface_hub but it was updated already

John6666 · June 20, 2025, 2:37pm

Perhaps

pip install -U huggingface_hub[hf_xet]

is better now for faster uploading / downloading with Xet.

jacek2024 · June 20, 2025, 2:40pm

Requirement already satisfied: huggingface_hub[hf_xet] in ./.pyenv/versions/3.12.3/lib/python3.12/site-packages (0.33.0)

is there any other way to download models? looks like I can’t download anything, should I use firefox again?

John6666 · June 20, 2025, 2:45pm

is there any other way to download models?

Oh… I believe the following method will cause a fallback to the normal connection. However, this will only work if the issue is caused by a bug in the library implementation or a misconfiguration on the library side.
Simply, if it’s the server’s fault, the only solution is for HF to fix the server settings!

pip uninstall hf_xet hf_transfer

jacek2024 · June 20, 2025, 2:48pm

I pressed ENTER few times:

huggingface-cli download --local-dir ~/models/ mradermacher/Skywork-SWE-32B-GGUF Skywork-SWE-32B.Q8_0.gguf
Downloading 'Skywork-SWE-32B.Q8_0.gguf' to '/home/jacek/models/.cache/huggingface/download/2Lnqyho0STxoIp7LIkr5NuMXlbA=.c34f43512c7d00ceed0d8d283c1c0a177927f96b8090cd691e3c4e1b6d5e3570.incomplete' (resume from 14416789809/34820885376)
Skywork-SWE-32B.Q8_0.gguf:   4%|██████▍                                                                                                                                                                               | 1.22G/34.8G [07:00<55:20:19, 169kB/s]


Skywork-SWE-32B.Q8_0.gguf:   4%|██████▋                                                                                                                                                                               | 1.28G/34.8G [17:35<60:32:08, 154kB/s]


Skywork-SWE-32B.Q8_0.gguf:   4%|███████▍                                                                                                                                                                              | 1.42G/34.8G [20:50<18:06:15, 512kB/s]
Skywork-SWE-32B.Q8_0.gguf:   4%|███████▉                                                                                                                                                                              | 1.51G/34.8G [26:30<28:56:31, 320kB/s]

John6666 · June 20, 2025, 3:01pm

Hmm… Is it being resumed…?

jacek2024 · June 20, 2025, 3:11pm

no, it’s always from 0, my point is that it’s very slow, but we will see I will wait few hours

John6666 · June 20, 2025, 3:18pm

Hmm… Last resort?

jacek2024 · June 20, 2025, 5:57pm

and it’s dead (forever?)

Skywork-SWE-32B.Q8_0.gguf:   4%|██████▍                                                                                                                                                                               | 1.22G/34.8G [07:00<55:20:19, 169kB/s]


Skywork-SWE-32B.Q8_0.gguf:   4%|██████▋                                                                                                                                                                               | 1.28G/34.8G [17:35<60:32:08, 154kB/s]


Skywork-SWE-32B.Q8_0.gguf:   4%|███████▍                                                                                                                                                                              | 1.42G/34.8G [20:50<18:06:15, 512kB/s]
Skywork-SWE-32B.Q8_0.gguf:  68%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌                                                          | 23.6G/34.8G [2:04:34<39:12, 4.75MB/s]
Skywork-SWE-32B.Q8_0.gguf:  68%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌                                                          | 23.6G/34.8G [2:04:51<39:12, 4.75MB/s]
Skywork-SWE-32B.Q8_0.gguf:  96%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉        | 33.3G/34.8G [2:33:21<01:57, 13.1MB/s]

jsulz · June 20, 2025, 7:12pm

Hi @jacek2024 thanks for the report and for following up on the GitHub issue as well.

Just a few questions to help troubleshoot this a bit more; could you answer a few questions/provide some more details:

Provide the output of pip freeze
Which repository you are downloading from? From the looks of the screenshots it is mradermacher/Skywork-SWE-32B-GGUF · Hugging Face is that correct?
What is the full command you are using?
Any details about the target environment (most notably what operating system you’re using any any information about network speeds and disk type)
Do you have any HF_XET environment variables set? (env | grep HF_XET should output them if they exist)

As a quick test, you might try to fall back to a HTTP download. You can do this through:

export HF_HUB_DISABLE_XET=1

And then rerunning your download commands. This might help unblock your current download while we troubleshoot the issue.

jacek2024 · June 20, 2025, 7:25pm

I tried to download 3 different models, same behaviour, it stops after 90% something and then freezes forever. I tried to download tiny model instead and it worked.
Now I am trying with this:

pip uninstall hf_xet hf_transfer

command is the same:

huggingface-cli download --local-dir ~/models/ mradermacher/Skywork-SWE-32B-GGUF Skywork-SWE-32B.Q8_0.gguf
Downloading 'Skywork-SWE-32B.Q8_0.gguf' to '/home/jacek/models/.cache/huggingface/download/2Lnqyho0STxoIp7LIkr5NuMXlbA=.c34f43512c7d00ceed0d8d283c1c0a177927f96b8090cd691e3c4e1b6d5e3570.incomplete'
Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
Skywork-SWE-32B.Q8_0.gguf:   2%|███▋                                                                                                                                                                                   | 703M/34.8G [03:08<4:11:16, 2.26MB/s]

jsulz · June 20, 2025, 7:46pm

Thanks for the command! Sorry, should’ve seen that was provided earlier; my apologies.

Few additional details that would be helpful:

pip freeze output
Environment info about where you are downloading to (i.e., what operating system you’re using any any information about network speeds and disk type)
Do you have any HF_XET environment variables set?
When the download progress stops, is there any network or CPU activity?

From your last comment, I’m wondering is if the issue is the chunk cache in the xet directory (present under the huggingface cache). If you clear the chunk cache (rm -rf ~/<cache_dir>/xet where <cache_dir> is the location of the Hugging Face cache) and then run export HF_XET_CHUNK_CACHE_SIZE_BYTES=0 that may help.

And of course, let me know if you are able to download via HTTP - thank you!

jacek2024 · June 20, 2025, 8:00pm

I use Linux, Ubuntu, fast nvme drives
I was downloading models with huggingface-cli many, many times before, so I am not doing anything new
I was removing whole .cache to clear the previous file
I don’t see any XET env

$ pip freeze
aiohappyeyeballs==2.6.1
aiohttp==3.12.13
aiosignal==1.3.2
annotated-types==0.7.0
anyio==4.9.0
attrs==25.3.0
certifi==2025.6.15
charset-normalizer==3.4.2
datasets==3.6.0
dill==0.3.8
distro==1.9.0
filelock==3.18.0
frozenlist==1.7.0
fsspec==2025.3.0
h11==0.16.0
httpcore==1.0.9
httpx==0.28.1
huggingface-hub==0.33.0
idna==3.10
jiter==0.10.0
multidict==6.5.0
multiprocess==0.70.16
numpy==2.3.0
openai==1.88.0
packaging==25.0
pandas==2.3.0
propcache==0.3.2
pyarrow==20.0.0
pydantic==2.11.7
pydantic_core==2.33.2
python-dateutil==2.9.0.post0
pytz==2025.2
PyYAML==6.0.2
requests==2.32.4
six==1.17.0
sniffio==1.3.1
toml==0.10.2
tqdm==4.67.1
typing-inspection==0.4.1
typing_extensions==4.14.0
tzdata==2025.2
urllib3==2.5.0
xxhash==3.5.0
yarl==1.20.1

jacek2024 · June 20, 2025, 11:12pm

Looks like everything works after uninstalling xet.
I successfully downloaded 3 big models.
What’s the point of this xet thing? What kind of “performance” I am losing?

Topic		Replies	Views
Cas-bridge.xethub.hf.co broke 🤗Hub	8	171	June 20, 2025
Snapshot_download repo getting stuck downloading, no error 🤗Hub	0	1427	August 17, 2023
Am I Black Listed By hugging face hub? 🤗Hub	0	1440	February 13, 2023
Huggingface Pytorch_model.bin download corrupted 🤗Hub	1	929	March 13, 2025
Solving Interrupted Downloads of Large Models on Hugging Face: Consistency Check Failure with deepseek-v3 Beginners	3	62	May 20, 2025

Strange problem with huggingface-cli

Related topics