Glad to hear that worked!
The primary benefit of Xet is faster uploads and downloads - you can see some of how this works on the infrastructure side here. On the client side it’s aided by providing huggingface_hub a tight integration with the xet-core Rust library that should provide some performance boosts on the download side especially - that’s hf-xet.
Obviously, in this case, something’s up!
Just trying to figure out what the issue was here so we can make sure to address this in the future; few additional questions if you wouldn’t mind:
- Are the NVMe drives you’re writing to local disks?
- What specific models in the repo were causing you problems?
- Did the downloads for these models stop at the same place each time?
- You mentioned you were clearing the
.cachedirectory each time - were you using thehuggingface-cli delete cachecommand to do this?