Hi folks, my apologies, I originally posted on this topic in the Beginners forum but since it concerns a project I’m looking for feedback and collaboration on it might be more at home here.
I have a FastAPI app (generated via LLM) which facilitates collaborative discussion using multiple Large Language Models.
Core Idea: I wanted to enable conversations between multiple LLMs, each potentially playing a specific role. This could leverage the strengths of specific LLMs while addressing any of their weaknesses with a complimentary “team member”.
Key Features:
Provider Abstraction: system integrates different LLM providers. Each provider is defined through a flexible payload template.
Dynamic Roles & Context: A key aspect of this app is the dynamic system prompt builder. This allows a user to assign specific roles to each LLM in a conversation, informing it of its responsibilities, its teammates, and its turn history. This promotes more coherent and collaborative outputs.
Conversation Orchestration: two main modes:
Step-by-Step: User manually guides the conversation flow, examining each LLM’s contribution in sequence.
Auto-Discuss: Enables automated, continuous conversations between LLMs based on a defined order, mimicking a collaborative brainstorming session. This continues until manually halted.
Rate Limiting and Resilience: integrated rate limiting to prevent API abuse and circuit breaker/retry logic to handle transient errors and API outages.
Data Management: The system stores conversation histories, roles, turn counts, and associated files in a database.
Export & Share: Conversations can be easily exported in JSON format for sharing and further analysis.
Streamlit UI: management of the API via web browser
TLDR:
This application provides a framework for exploring multi-agent LLM collaboration. It’s designed to allow you to:
Define LLM “agents” with specific roles and personalities.
Set up conversational workflows (manual or automated).
Observe how these agents interact and generate results.
I believe the app provides a valuable foundation for exploring multi-agent LLM systems. It’s a starting point for making more sophisticated and intelligent collaborative AI experiences available to users who might prefer to operate via UI.
I would love to share it and make it open source, but I would first like to develop it further- while the prototype works as explained above, I would love to collaborate on this with people who actually know what they’re doing (I’m still learning).