Error Uploading Large Folder

Facing this error:

Your push was rejected because it contains files larger than 10 MiB. Please use https://git-lfs.github.com/ to store large files. See also: https://hf.co/docs/hub/repositories-getting-started#te

I am using: hf upload-large-folder from CLI. Before I was able to upload my files. I wonder if I hit some limit on the repo.

1 Like

This error appears when Git LFS is not installed. Installation is very simple on Linux and Mac.

On Windows, you need to download and install both Git and Git LFS from installers.

For Windows

Git lfs is installed. When uploading to another dataset the code works …

1 Like

When uploading to another dataset the code works …

Hmm, that’s strange…

In the case of quota-related errors, error messages tend to appear that indicate the cause as much as possible (such as “Rate Limited”, “429”, “Quota Exceeded”, “Offending Files”, etc.).

It may really be some kind of restriction…? :thinking:

By the way, I recommend updating hf_xet just in case.

pip install -U huggingface_hub[hf_xet]

Yes, there is an offending file referenced in the error: the .gitattributes.

Idk if this make sense to you

1 Like

Even if specified in .gitattributes, it seems impossible to avoid…:thinking:

Other than that, an error occurs when the file size is zero.
Hmm… It seems difficult to pinpoint the cause based on speculation alone. Setting the HF_DEBUG environment variable to 1 allows you to see detailed error messages.