Constant 503 error for several days when running LLAMA 3.1

Inference Endpoints on the Hub
1

Hi all. Earlier this month I wrote a script incorporating LLAMA 3.1, and it worked well (with timeouts, etc), but for the past week whenever I try to access LLAMA 3.1 at all I get a 503 error (including for short simple requests, like HF’s sample code).

Any idea what the problem may be? I do indeed have Pro.

1 Like
2

Did you find a solution to this problem? I am experiencing the same problem on and off for a couple of days now.

1 Like
3

There were at least three reports of the same situation on Discord, so I think it’s probably some kind of error. @michellehbn

4

Am still getting a 503 when running Llama-3.2-3B-Instruct. Any known solution?
huggingface_hub.errors.HfHubHTTPError: 503 Server Error: Service Temporarily Unavailable for url: https://router.huggingface.co/hf-inference/models/meta-llama/Llama-3.2-3B-Instruct

1 Like
5

Depending on the model, it seems that this can be avoided by the following methods.

1 Like
6

Thanks. Using providers works like magic. Here is the documentation: HF Inference
Sample

client = InferenceClient(
    provider="hf-inference", 
    token=os.environ["HF_TOKEN"])
1 Like