Dynamically mount a Gradio `Block` from within a FastAPI function

🔒 Gradio
1

I’d like to dynamically create and mount Gradio blocks in a FastAPI app. This doesn’t seem to work too well though…what am I missing? Here’s a minimal example that doesn’t execute as expected:

app = FastAPI()

@app.get('/test')
def test(procId: int):
    with gr.Blocks() as demo:
        chatbot = gr.Chatbot()
        msg = gr.Textbox()
        clear = gr.Button("Clear")

        def user(user_message, history):
            return "", history + [[user_message, None]]
        
        def bot(history):
            bot_message = random.choice(["I really hope this works", "Gradio is fun"])
            history[-1][1] = bot_message

        msg.submit(user, [msg, chatbot], [msg, chatbot], queue=False)\
            .then(bot, chatbot, chatbot)

    global app 
    demo.queue()
    app = gr.mount_gradio_app(app, demo, f'/gradio/{procId}')

    return f'Gradio app mounted at /gradio/{procId}'

if __name__ == '__main__':
    uvicorn.run('test:app')

I can call /test/?procId=100 and it runs fine. However, when I go to /gradio/100 the chatbot UI is there, but it doesn’t appear like any of the functionality works. I can type into the message box, but when I press enter the chatbot doesn’t respond with anything. Does this have something to do with test() being called on a separate Thread and terminating before I can load the gradio endpoint?

2

Ok it seems like the main problem is with demo.queue(). That is, gradio can’t mount a streaming output interface without the functionality breaking. I’m hoping there’s a way around this though.

3

What version are you using? I get a different error when running your code