Hi!
I try to use the following python code to upload my own dataset:
from huggingface_hub import HfApi
api = HfApi()
api.upload_large_folder(repo_id=“hhyyjj/egoexo”, repo_type=“dataset”,folder_path=“Downloads/processed_xmem”)
Initially, everything worked well, but after uploading about 96GB of data, I encountered an error. Here’s a screenshot of the error message:
Does anyone know how to resolve this issue? I’d greatly appreciate any advice or solutions!
Thanks in advance!
Although it is not prohibited, there is a possibility that it will become unstable. The following function seems to be more stable when uploading large folders.