SALLMA: A Software Architecture for LLM-Based Multi-Agent Systems

As a new and disruptive technology, the introduction of large language models (LLMs) may be the first step into a paradigm shift of how we develop and deploy software-intensive systems. While the capabilities of LLM agents for software engineering and architecture tasks are currently explored, how to architect LLM-based systems appears to be to date an uncharted territory. Software architectures based on a single LLM agent face inherent challenges, such as lack of task customization, lack of memory, and limited access to ground truth. These challenges become especially pressing in real-world contexts that demand persistent context, validated information, and task-specific flexibility. As a potential solution to overcome these challenges, multiple LLM-agents can be adopted for specialized tasks within a single software-intensive system. In this contribution, we open the discourse on architecting LLM-intensive software products by presenting SALLMA, a Software Architecture for LLMbased Multi-Agent systems. SALLMA leverages two core layers, namely (i) the Operational Layer, responsible for request intent management, handling real-time task execution and dynamic orchestration of agents, and (ii) the Knowledge Layer, used to to store and manage metamodels and configurations for workflows and agents. To primarily assess the viability of SALLMA, we develop a proof of concept leveraging as key technologies Docker, Kubernetes, Python, LangChain, Hugging Face, Mistral, LLaMA, and SQL and NoSQL databases. Currently, SALLMA is deployed to provide information on behalf of public administration offices, and is currently utilized in a business simulation scenario.