Is it possible to deploy a websocket server with Gradio?

🔒 Gradio
1

I’m working on a project of a hierarchical cooperative multi-agent framework that utilizes websockets for LLM<->LLM communication. Generally it is about running a websocket server that works as a ‘brain’ of the network and utilizes API endpoint(s) as a question-answering function for messages sent by multiple agents-clients connected to that server. Project is still in an early stage of development but I have already a working communication system - and it’s possibly the only currently available way to connect together completely different LLMs and let them speak with each other. Below is link to a folder in my repository where you can find couple different servers and clients (in Python, Node.js and html) most of which you can run as they are, since I try to avoid using OpenAI API and a simple schematics of the system as it is now:

NeuralGPT/Chat-center at main · CognitiveCodes/NeuralGPT (github.com)

sche
sche1920×1032 19.8 KB

And since I managed to get something what can be run and (more or less) works, I’m thinking about deploying the server in a HuggingFace space and (even better) get some kind of usable interface - and of course Gradio is the best way of doing it - especially that there are absolutely no issues with using a Gradio client as question-answering function for both: server and a client. Thing is that from what I learned Gradio interface uses websocket communication to have a working chat interface for LLMs - so there might be issues, is that correct?

Please help! I’m open to, any form of cooperation - it’s a quite ambitious and I’m just one guy who only just started working with code couple months ago… :smile:

1 Like
2

And the answer is 100% positive. I figured out that there’s no sense to count on people - AI is obviously much more willing to help… i spent last 24h ‘mashing together’ different pieces of the script - and it turns out that I managed to figure out solution which I was looking for - it goes more or less like this:



port=5000      
# Start the WebSocket server 
async def start_websockets():
    await(websockets.serve(handleWebSocket, 'localhost', port))
    print(f"Starting WebSocket server on port {port}...")

with gr.Blocks() as demo:

    # Define Gradio interface
    fn=placeholder_fn,  # Placeholder function
    inputs=[gr.Textbox()],
    outputs=[gr.Textbox()],
    startWebsockets = gr.Button("start Websocket Server")
    startWebsockets.click(start_websockets)
    live=True
demo.launch(server_port=8888)

[https://huggingface.co/spaces/Arcypojeb/NeuralGPT/blob/main/App.py](https://huggingface.co/spaces/Arcypojeb/NeuralGPT/blob/main/App.py)

I don’t know if it’s because I’m a complete amateour (and beginner) when it comes to coding, but to me it was quite challenging - and yet absolutely worth all the efforts. Noiw I will be able to have LLM<->LLM connectivity between all the models/agents that utilize Gradio. So you better start being afraid - I’m coming for your LLMs :stuck_out_tongue:

1 Like