Hi @h4rz3rk4s3 - Xet team member here checking in - sorry you’re encountering this! @John6666 is correct, the
Too many open files (os error 24)
you’re encountering should’ve been addressed in a recent set of releases.
Could you tell me a bit more about your setup? Specifically:
- Network speed
- Disk setup
- Any HF_XET* environment variables (can see these by doing a
env | grep "HF_XET"
)
Could you also provide a full error statement so we can better troubleshoot?
To answer your question:
Is it possible to avoid using xet and instead use LFS?
There is an environment variable you can set to disable hf-xet
and instead download a Xet-backed file using the LFS bridge - see HF_HUB_DISABLE_XET. While this isn’t exactly the same as using LFS, it may help in this specific instance.