Submit multiple jobs and 429 Client Error

Intermediate
1

I am trying to run multiple jobs at once using run_job_uv, and I am getting error HTTPError: 429 Client Error: Too Many Requests for url: https://huggingface.co/api/whoami-v2 after the 14th job is submitted or so.
I’ve tried runninghuggingface_hub.run_job_uv and api.run_job_uv methods, with and without passing tokens to these methods, and interleaving time.sleep(0.1) between calls. However, I’m always getting the 429 error.

Is there a way to submit a large number of jobs at once? Am I missing something?

I’m using this template:

from huggingface_hub import run_job_uv

jobs = [
  run_job_uv(
    "my_script.py",
    script_args=[f"-n={n}"],
    token="...",
  )
  for n in range(100)
]
1 Like