Intermittent 500s on Inference Endpoint - requests not reaching the container

Inference Endpoints on the Hub
1

I’m using a personal account. I’m running a custom-handler Inference Endpoint for text embeddings (Llama-3.1-8B). Since early this morning on 2026-07-14, I’m getting intermittent HTTP 500s (~70% of requests fail, ~30% return 200 normally).
What I’ve ruled out:

Not GPU-specific - happens on both L40S and A100.
Not endpoint-specific - brand-new endpoints show the same.
Not my code/deps - the same build ran hundreds of thousands of requests fine yesterday.
Container is healthy - model loads successfully, and successful requests return correct results in 30–415 ms.

Key finding: the container logs only show the successful requests. The failed (500) requests don’t appear in the container logs at all — they seem to be failing at the gateway/router layer before reaching the container.
The status page shows no incident. Is there a known issue with the us-east-1 inference gateway, or something on my account causing this? Happy to share endpoint IDs privately.

Thanks!

2

Same exact thing for me with a us-east T4; began almost an hour ago.

3

Same here.

4

I deployed the same node in eu-west-1 and it’s functional, so I’m guessing aws is having some type of outage in us-east.

5

Hey all! Thanks for reporting. We’re actively investigating and I’ll update you here when it’s fixed.

6

Thanks for waiting - the 500 errors were related to recent infrastructure work today and we’re sorry for the scare! We applied a fix though please let us know if you continue running into issues.

Thank you for bearing with us as well :hugs: