I just found this thread after getting rate-limited under these circumstances (huggingface-cli upload-large-folderon a dataset with lots of small files). I understand it’s still a feature-under-construction and all and I just wondered when should I try to resume (I assume running huggingface-cli upload-large-folder again will resume, right?). The error message said to wait about 1h, is this indeed correct?
UPDATE: I just switched it to use git-lfs directly and so far it looks like it’s working (as in uploading). Also, it might be that upload-large-folder failed because I had > 10k file in a single folder? (Says here it’s a bad idea)
Thank you @Wauplin