Understanding ensemble-based component architectures by LLMs

Abstract Ensemble-based component systems have been used for many years to develop collective adaptive systems (CAS). The DEECo component model offers a framework for modeling and implementing ensemble-based component systems. Being expressive enough and having semantics specifically tailored towards dynamically evolving systems, DEECo has proven to be fairly powerful in modeling complex and dynamic architectures. We see great potential in employing large language models (LLMs) to simplify creating and refining the DEECo architectures. Since this constitutes a large research scope, in this paper, we focus on initial experiments to demonstrate how well generic LLMs (two OpenAI models executed remotely and four open-source models executed locally) understand the advanced concepts of ensemble-based CAS embodied in DEECo. We do so by systematically asking six questions about specific details of three DEECo applications that differ in the way they are specified. Our results indicate that LLMs can indeed understand ensemble-based architectures and show how this is influenced by the specification means. In particular, using external DSL, which is very self-explanatory, gave good results out of the box. Specifications embedded in existing programming languages needed a prior explanation of how to interpret them.