I currently have an endpoint set up for Google’s Flan-UL2. Recently, I added multithreading to my code to allow it to send multiple requests to the server at once. When I did that, however, the endpoint began sending a number of 502 - Bad Gateway responses.
I was wondering what the 502 Bad Gateway responses mean, and what the source of this error might be? Interestingly, I was able to troubleshoot the code currently by having each thread wait 1 second and then resend the request. The second request does almost never gives a bad gateway response.
I also checked the log files to see if there was any sign of the bad gateway response, but did not see any.