Creating A Team Of LLMs

Hi folks, my apologies, I originally posted on this topic in the Beginners forum but since it concerns a project I’m looking for feedback and collaboration on it might be more at home here.

I have a FastAPI app (generated via LLM) which facilitates collaborative discussion using multiple Large Language Models.

Core Idea: I wanted to enable conversations between multiple LLMs, each potentially playing a specific role. This could leverage the strengths of specific LLMs while addressing any of their weaknesses with a complimentary “team member”.

Key Features:

Provider Abstraction: system integrates different LLM providers. Each provider is defined through a flexible payload template.

Dynamic Roles & Context: A key aspect of this app is the dynamic system prompt builder. This allows a user to assign specific roles to each LLM in a conversation, informing it of its responsibilities, its teammates, and its turn history. This promotes more coherent and collaborative outputs.

Conversation Orchestration: two main modes:

Step-by-Step: User manually guides the conversation flow, examining each LLM’s contribution in sequence.

Auto-Discuss: Enables automated, continuous conversations between LLMs based on a defined order, mimicking a collaborative brainstorming session. This continues until manually halted.

Rate Limiting and Resilience: integrated rate limiting to prevent API abuse and circuit breaker/retry logic to handle transient errors and API outages.

Data Management: The system stores conversation histories, roles, turn counts, and associated files in a database.

Export & Share: Conversations can be easily exported in JSON format for sharing and further analysis.

Streamlit UI: management of the API via web browser

TLDR:
This application provides a framework for exploring multi-agent LLM collaboration. It’s designed to allow you to:
Define LLM “agents” with specific roles and personalities.
Set up conversational workflows (manual or automated).
Observe how these agents interact and generate results.

I believe the app provides a valuable foundation for exploring multi-agent LLM systems. It’s a starting point for making more sophisticated and intelligent collaborative AI experiences available to users who might prefer to operate via UI.

I would love to share it and make it open source, but I would first like to develop it further- while the prototype works as explained above, I would love to collaborate on this with people who actually know what they’re doing (I’m still learning).

1 Like

Currently, what I’ve been able to use the application for includes brainstorming, with each LLM assigned to a particular task or to surface a specific perspective, role-play (each LLM embodies a different character from a story, playing out an interaction between them, or having each LLM embody a different historical figure and querying them) and experimenting with model behavior where I put several of them together in the auto-discuss environment and tell them they are free to discuss anything they would like. I’m curious about if this could be used in some way to have LLMs train LLMs, like a sort of auto RLHF or distillation?

What would be the easiest way to get this up and running on a space here? I apologize if I’m not asking this question in the right place- there are a couple angles of this sort of thing I understand pretty well, and many other angles I have absolutely no knowledge of. I’m just excited by my ideas and would love to create and share something neat!

1 Like

I’m not familiar with the theory of generative AI, but there are attempts at things like arenas and orchestration. I wonder if yours is similar to a deliberative system.

1 Like