How to get a list of all Huggingface download redirections to whitelist?

🤗Hub
1

I work inside a secure corporate VPN network, so I’m unable to download Huggingface models using from_pretrained commands. However, I can request the security team to whitelist certain URLs needed for my use-case.

The security team has already whitelisted the ‘huggingface.co’ and ‘cdn-lfs.huggingface.co’ URLs. I can now download the files from repo but the loading functions from_pretrained still don’t work.

I think it’s getting blocked while redirecting the requests internally. So, is there a way to know all (hop) URLs I can request to whitelist to make the load functions work?

Thanks in advance.

10 Likes
2

hi @ayadav

Can you give more details, like error logs, etc?

3

Is there any update on this?

1 Like
4

Having the same issue. Is there a listing of URLs that we can whitelist? Also if there are any planned changes to URLs is there a roadmap so we can stay on top of it?

1 Like
5

I’ll try to supply error logs next time I encounter it, but it has come up multiple times for me as well. When we try to call <model>.from_pretrained("repo") in our DataBricks environment, we get an SSL error about not having the proper certificate. We’ve also gotten a max_retries error but I can’t say for certain if that was due to the underlying whitelist request. There are ways around this, but if HF published a domain list that we could use to properly configure our environments, that would be very useful!

2 Likes
6

hi! any updates on this? or any alternatives to follow meanwhile? I am about to try downloading a model and going offline and then pushing it up to databricks. Yet, if you had a better idea, or tried this before, I’d like to hear.

7

I have same issue with download from different cdn name.
After our IT team added
http://huggingface.co/ and
http://cdn-lfs.huggingface.co/ in whitelist.

For example, it is work for download meta-llama/Llama-2-13b-chat.
But error when the cdn become cdn-lfs-us-1.huggingface.co or other regions.

8

Update? Same issue here. I’ve gotten around by using my home network to connect to the hf repo and download to my workstation cache. Then I reconnect to VPN into the corporate network and copy from my workstation to the server cache. This is painfully slow.

FWIW curl -IL test shows redirection (302 responses) from the repo when I am connected to the corporate network (fails to download). However on my home network there are no redirects (successful download). Is there an issue with general redirection handling?

9

Hey was anyone able to find a solution for this?

10

Related:

11

Note that for security reasons, we recently updated the domain for our CDN; in order to be able to download files you also need to whitelist the following domains:

9 Likes
12

we have created exception for SSL inspection for FQDN listed by pierric plus these 2 ones:

But it is still does not work, always same error encountered SSL: CERTIFICATE_VERIFY_FAILED when trying to download sentence-transformers/all-MiniLM-L6-v2

1 Like
13

Hi @pierric has the above list changed since the XetHub announcement?

While downloading, I’m seeing a domain of cas-bridge.xethub.hf.co as well. Is this the only additional domain or are there others?

1 Like
14

Hey @sean-pai, sorry about that, indeed we recently started migrating some repos from LFS to Xet (checkout this blogpost if you want to learn more about Xet).

As a result (and as you found out), you need to add cas-bridge.xethub.hf.co for the download path (I updated my original reply above). We’ll communicate here when we enable the Xet upload path.

3 Likes
15

Hi @sean-pai, just a quick follow up, we’ve just released the Xet client which can be used to download these repos using the xet format directly. If you are interested in faster downloads of Xet enabled repos, follow these instructions here.

If you install the client and download the same content, you will also need to add two new endpoints, cas-server.xethub.hf.co and transfer.xethub.hf.co.

1 Like
16

Hi @brianronan,

The certificate returned for cas-server, is the cas-bridge certificate.

(.venv) mark@wide:~/prog/b3d-lora-trainer$ openssl s_client -connect cas-server.xethub.hf.co:443 -servername cas-server.xethub.hf.co

Connecting to 52.71.209.178
CONNECTED(00000003)
depth=2 C=US, O=Amazon, CN=Amazon Root CA 1
verify return:1
depth=1 C=US, O=Amazon, CN=Amazon RSA 2048 M03
verify return:1
depth=0 CN=cas-bridge.xethub.hf.co
verify return:1

Certificate chain
0 s:CN=cas-bridge.xethub.hf.co
i:C=US, O=Amazon, CN=Amazon RSA 2048 M03
a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
v:NotBefore: Jan 29 00:00:00 2025 GMT; NotAfter: Feb 27 23:59:59 2026 GMT
-snip-

And thus I get certificate verify failed when using from_pretrained().

model_name = "Qwen/Qwen2.5-Coder-7B"
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    trust_remote_code=True,
    torch_dtype=torch.float16,
    device_map="auto"
)

“timestamp”:“2025-05-26T17:43:40.209499Z”,“level”:“WARN”,“fields”:{“message”:“Reqwest(reqwest::Error { kind: Request, url: "https://cas-server.xethub.hf.co/reconstruction/cd9b3569e15af48b5338d1f02bf99476542809310dde89f1a4301215b1a8a81d\”, source: hyper_util::client::legacy::Error(Connect, Ssl(Error { code: ErrorCode(1), cause: Some(Ssl(ErrorStack([Error { code: 167772294, library: "SSL routines", function: "tls_post_process_server_certificate", reason: "certificate verify failed", file: "ssl/statem/statem_clnt.c", line: 2092 }]))) }, X509VerifyResult { code: 20, error: "unable to get local issuer certificate" })) }). Retrying…“},“filename”:”/home/runner/work/xet-core/xet-core/cas_client/src/http_client.rs",“line_number”:175}

1 Like
17

Just noting for the followers of this thread that the issue raised here by @marked23 is being handled over here - Certificate Verify Failed cas-server vs. cas-bridge · Issue #351 · huggingface/xet-core · GitHub - and currently seems unrelated to any issues around whitelisting domains.

1 Like
18

This was working for us but recently started failing with timeouts whenever we use huggingface_hub (via python or CLI).
I noticed we can still download using curl -L https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/model.safetensors?download=true --output model.safetensors but we cannot using

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')

Nor using

huggingface-cli download sentence-transformers/all-MiniLM-L6-v2

Both of these just hang like:

huggingface-cli download sentence-transformers/all-MiniLM-L6-v2 --max-workers 1
Fetching 30 files:   0%|                                                                                                                                                                                                                           | 0/30 [00:00<?, ?it/s]Downloading 'model.safetensors' to '/home/jupyter/.cache/huggingface/hub/models--sentence-transformers--all-MiniLM-L6-v2/blobs/53aa51172d142c89d9012cce15ae4d6cc0ca6895895114379cacb4fab128d9db.incomplete'

model.safetensors:   0%|                                                                                                                                                                                                                      | 0.00/90.9M [00:00<?, ?B/s]
"timestamp":"2025-07-01T13:40:33.080005Z","level":"WARN","fields":{"message":"Reqwest(reqwest::Error { kind: Request, url: \"https://cas-server.xethub.hf.co/reconstruction/789fdf16a3e59f4fbfb6002967ecee539a198dadb5be74ca549aa7dc9b1b55fb\", source: hyper_util::client::legacy::Error(Connect, ConnectError(\"tcp connect error\", Os { code: 110, kind: TimedOut, message: \"Connection timed out\" })) }). Retrying..."},"filename":"/home/runner/work/xet-core/xet-core/cas_client/src/http_client.rs","line_number":200}
{"timestamp":"2025-07-01T13:40:33.080067Z","level":"WARN","fields":{"message":"Retry attempt #0. Sleeping 2.851275886s before the next attempt"},"filename":"/root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/reqwest-retry-0.7.0/src/middleware.rs","line_number":171}
{"timestamp":"2025-07-01T13:58:03.703922Z","level":"WARN","fields":{"message":"Reqwest(reqwest::Error { kind: Request, url: \"https://cas-server.xethub.hf.co/reconstruction/789fdf16a3e59f4fbfb6002967ecee539a198dadb5be74ca549aa7dc9b1b55fb\", source: hyper_util::client::legacy::Error(Connect, ConnectError(\"tcp connect error\", Os { code: 110, kind: TimedOut, message: \"Connection timed out\" })) }). Retrying..."},"filename":"/home/runner/work/xet-core/xet-core/cas_client/src/http_client.rs","line_number":200}
{"timestamp":"2025-07-01T13:58:03.703998Z","level":"WARN","fields":{"message":"Retry attempt #1. Sleeping 2.339135315s before the next attempt"},"filename":"/root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/reqwest-retry-0.7.0/src/middleware.rs","line_number":171}

It just hangs and times out for the model.safetensors file.

We have allowlisted:

cdn-lfs-us-1.hf.co
cdn-lfs-eu-1.hf.co
cdn-lfs.hf.co
cas-bridge.xethub.hf.co

Any ideas?
It seems to be going to a cloudfront IP at some point, but I do not know what for and if it is something that can be stopped.

1 Like
19

Hi @mariovela

Could you try allowlisting the following URLs in addition to the current domains you’ve allowlisted:

transfer.xethub.hf.co
cas-server.xethub.hf.co

Both are used when downloading from/uploading to Xet-enabled repositories when hf-xet is installed.

See @brianronan’s comment above

1 Like
20

My bad! That works! Thank you! :smiley:

2 Likes