502 Bad Gateway Error for Flan-UL2 model

Inference Endpoints on the Hub
1

Hello!

I currently have an endpoint set up for Google’s Flan-UL2. Recently, I added multithreading to my code to allow it to send multiple requests to the server at once. When I did that, however, the endpoint began sending a number of 502 - Bad Gateway responses.

I was wondering what the 502 Bad Gateway responses mean, and what the source of this error might be? Interestingly, I was able to troubleshoot the code currently by having each thread wait 1 second and then resend the request. The second request does almost never gives a bad gateway response.

I also checked the log files to see if there was any sign of the bad gateway response, but did not see any.

Thanks!

2

Have you checked the logs?

3

Yep! I looked through the logs but I didn’t see any signs of errors or reasons for sending a 502. Should the logs indicate why the 502 error occurred, or is there another place I can find that information?