I created space using Gradio, because I really need the API functionality. In app.py I am simply loading OpenAI’s whisper and getting predictions from audio. But always, when the request takes longer than a minute I get 504. No matter what I do. I tried setting enable_queue param (but that’s enabled by default), using the queue method, setting timeouts on the client, changing clients (from flutter http to python requests) and using fastapi in gradio space. Nothing worked so far. The error is from nginx so I think there is some set timeout, that I cannot change. MAYBE I could use set command to change the /etc/… nginx conf from simple subprocces command, but I am not sure that that would work and I also think it should work by default or be easily changable. I would really appreciate some response since I’ve been struggling with this for a few days now.
Yes of course. Sorry I forgot that. Here: OpenAIWhisperLargePredictions - a Hugging Face Space by Skyrr
Right now I am using FastAPI and the API address is:
https://hf.space/embed/Skyrr/OpenAIWhisperLargePredictions/notes
Thank you!
The API seems to reply for me.
Though your endpoint seems to require a body and you defined a GET method for /notes
Changing @app.get("/notes") to @app.post("/notes") should allow you to post to this endpoint
So it replies for you even when it takes longer than a minute?
I used post with gradio’s default API. The problem was still there. But I can try post with FastAPI.
Tried that, still getting the same error. Gateway timeout after a minute.
Could you provide a sample body so that I can try the requests on my side.
It seems your call triggers a 500 on your api, might be linked
Sure thing.
Flutter:
final url = Uri.parse('https://hf.space/embed/Skyrr/OpenAIWhisperLargePredictions/+/api/predict');
final response = await http.post(url,
headers: {
'Content-Type': 'application/json',
'Accept': 'application/json',
},
body: jsonEncode(<String, List<dynamic>>{
"data": [
{"name": fileName, "data": readAudioFile(path)}
]
}),encoding: Encoding.getByName('utf-8'))
.timeout(const Duration(seconds: 60 * 60));
String readAudioFile(String path) {
final file = io.File(path);
List<int> bytes = file.readAsBytesSync();
String base64String = base64Encode(bytes);
final fileString = 'data:audio/wav;base64,$base64String';
return fileString;
}
Python requests:
import requests
import base64
def main():
url = "https://hf.space/embed/Skyrr/OpenAIWhisperLargePredictions/notes"
headers = {
"Content-Type": "application/json",
"Encoding": "utf-8",
"Accept": "application/json",
}
data = {"audio": base64decode(r"C:\Users\skirl\Downloads\1664830934891.wav")}
response = requests.post(url, headers=headers, json=data,timeout=3600)
print(response.text)
def base64decode(path):
return base64.b64encode(open(path, "rb").read()).decode("utf-8")
if __name__ == "__main__":
main()
Please note that I didn’t yet update the url in flutter.
So there is indeed a timeout after 1 min.
Within gradio when using the UI the call is not done directly but a websocket is open and the call down internally, the result is then sent to the websocket
Any ideas how to solve or bypass that?
The timeout is actually expected to prevent requests from hanging forever in the event of the space is crashed and cannot restart.
Using the gradio interface directly, do you also experiment the 504 or only when directly querying the API ?
Only with the API. Interface works fine. Are you saying that I can open a socket connection, send request and just listen for response?
Yes, it seems that’s what gradio is doing
Okay. Thank you so much for helping me solve this.
Hi @Skyrr ! Thank you for posting and thank you @chris-rannou for helping.
If you want to use a space programmatically from another space, the best way is the load method on gradio.Blocks.
So if you do something like other_app = gr.Blocks.load("spaces/<your-space>") , you can use other_app like a regular python function. For example , output = other_app(<input>, fn_index=0). This will take care of creating the websocket for you.
Here is an example using this approach to call stable diffusion from another app: Using loaded interface as a function in Blocks return error · Issue #2369 · gradio-app/gradio · GitHub
Here is a guide talking about this more in depth: Using Blocks Like Functions
Thank you for suggestion. The problem is that I need to connect to the app from flutter. I actually decieded that easiest option will be to just use whisper-large repo directly with inference api. Only issue there is that it’s often overloaded. I will se how to solve that. Maybe later I’ll use inference endpoints.
Have you solved the problem yet? I’m having a 504 time out problem. When I request input to space, the result returned is as follows:
b’\r\n504 Gateway Time-out\r\n\r\n
504 Gateway Time-out
\r\n\r\n\r\n’.If you have solved it please help me. Thank you
