429 Error from Amazon SageMaker Classic Notebook

🤗Transformers
1

When attempting to train timm models using fastai library in a SageMaker Classic Notebook (ml.g4dn.4xlarge), I continue to get 429 errors when creating the learner. This has happened for both:

HfHubHTTPError: 429 Client Error: Too Many Requests for url: https://huggingface.co/timm/convnext_base.clip_laion2b_augreg_ft_in12k_in1k_384/resolve/main/model.safetensors

and

HfHubHTTPError: 429 Client Error: Too Many Requests for url: https://huggingface.co/timm/convnextv2_base.fcmae_ft_in22k_in1k_384/resolve/main/model.safetensors

How do I fix this to not get blocked by HfHub?

2

This happend with us too, I think you can just logout for a shile from AWS may be stop your instance and restart it, issue will go away, give it some time. I dont know why this happens, but i did the same and was able to work then

3

I noticed the same behavior. Restarting the sagemaker notebook instance does NOT always work. I think it ultimately is related to the HF API endpoint for /api/whoami-v2 GET, which is likely doing some overly aggressive filtering based on the IP address of AWS resources. It’s not a great solution, but switching to another cloud provider (Lambda Labs) worked for me. I suspect if you spun up a Sagemaker Notebook instance in a lesser used region (i.e., outside of us-east-1), you might have better luck.