Downloading larger models with xet fails on macOS

Hi @h4rz3rk4s3 - Xet team member here checking in :hugs: - sorry you’re encountering this! @John6666 is correct, the Too many open files (os error 24) you’re encountering should’ve been addressed in a recent set of releases.

Could you tell me a bit more about your setup? Specifically:

  • Network speed
  • Disk setup
  • Any HF_XET* environment variables (can see these by doing a env | grep "HF_XET")

Could you also provide a full error statement so we can better troubleshoot?

To answer your question:

Is it possible to avoid using xet and instead use LFS?

There is an environment variable you can set to disable hf-xet and instead download a Xet-backed file using the LFS bridge - see HF_HUB_DISABLE_XET. While this isn’t exactly the same as using LFS, it may help in this specific instance.

1 Like