I uploaded a model to huggingface, it has a pytorch_model.bin that is 12.2gb large, the sha256 is the same for website and locally but when i download it again, it is 11.7gb sha256 is different and its unopenable. I can confirm this behavior happens on a direct download, in kobold-ai colab and kobold-ai local. I couldn’t find anyone else having this issue, any ideas?
1 Like
I have also faced this issue multiple times…
possible solutions:
Git LFS (Large File Storage):
-
Hugging Face uses Git LFS for large files. Ensure you have Git LFS installed and configured properly. You can check by running
git lfs install
and then cloning the repository again. -
- Hugging Face provides a faster transfer tool called
hf_transfer
. You can enable it by installing thehuggingface_hub
library and setting the environment variableHF_HUB_ENABLE_HF_TRANSFER=1
. This might help with large file downloads.
- Hugging Face provides a faster transfer tool called
-
- Instead of downloading directly, try using the Hugging Face Python library:
python
from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="your_repo_id", filename="pytorch_model.bin")
Replace "your_repo_id"
with the actual repository ID.
1 Like