Hi guys, I’ve been trying to upload 65b models using huggingface_hub’s API (upload_folder). I plan to upload different stages of checkpoints to different branches. I successfully uploaded to a few branches but started to fail and here’s the error message:
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: https://huggingface.co/api/complete_multipart?uploadId=5BrbWlZ7XvfMEvePpHubm255Af8U6zOOs9cbHrTbR0sla2pl0O_NQSiK87ge2K1pHGX1SuA8Q0MsQLyW25WA5xHXCwRftC4xrTKFiLUCu4iLxQWYz7A.CmEQdylXhuLn&bucket=hf-hub-lfs-us-east-1&prefix=repos%2F9e%2Fa6%2F9ea61a7c40e45d7749c396716d3ae182516ee10decd7eb7b7fbcd922fe6f6245&expiration=Tue%2C+05+Mar+2024+21%3A18%3A17+GMT&signature=7830b619b655aa29bf0f7d955cbe67f75daddefb5876d04cf9fb3393d6cdfcd8
here’s how I use the api:
api.upload_folder(
folder_path=…,
repo_id=…,
repo_type=“model”,
revision=…,
)
I also tried to set multi_commits=True, but it only works with main branch. Any suggestion on how to work around this? Thanks in advance!