DYNAMIC MECHANISMS AND METRICS IN LANGUAGE MODEL-BASED MULTI-AGENT SYSTEMS: A SCOPING REVIEW

Background. Multi-agent systems based on large language models (LLMs) have become popular as architectures for complex orchestration tasks, but the concepts that inspire them—self-organization, nonlinear dynamics, information theory, and nonequilibrium thermodynamics—come from complex systems and classical swarm intelligence, and their validity in the LLM domain has not been systematically established. Objective. To map the available evidence on self-organization mechanisms, nonlinear dynamics, information theory, and nonequilibrium thermodynamics applied to LLM-based multi-agent systems, as well as the observables and metrics used to characterize their collective behavior and the conditions under which these concepts could be transferred to this domain. Methods. Scoping review following the JBI methodological guidance and the PCC framework. Searches were conducted in OpenAlex, arXiv, and Scopus. Two independent reviewers—language model instances with human verification—screened the records; disagreements were resolved through adjudication with a third reviewer and human review. Data were extracted with a standardized 16-domain instrument, and each study was classified according to its evidence proximity to the LLM domain (direct evidence, plausible transfer, or analogical transfer). Results. A total of 229 studies were included. Only 16 (7.0%) constitute direct evidence on LLM-based multi-agent systems, and only 3 of them report quantitative metric values. Transferable evidence from related systems is abundant (163 non-direct studies with at least one evaluable thematic block), but its availability and consistency vary across constructs. None of the six constructs evaluated achieves full empirical support; four show partial support (S2-S5) and two remain non-evaluable (S1, S6). Reproducibility reporting is limited: seed, number of runs, and sampling frequency are not reported in any study. A full-text audit of the 16 direct studies confirmed a reporting asymmetry: descriptive fields (model, dataset, task, and baseline) are reported in all 16 studies, whereas seed and sampling frequency are not reported in any, number of runs in 3, and temperature in 1. A pre-registered contrast between field families (median difference 0.90; 95% CI [0.80; 1.00]; exact permutation test p = 0.015) is consistent with a systematic reporting asymmetry under the prespecified operationalization. Conclusions. The literature provides a broad repertoire of mechanisms, observables, and metrics potentially transferable to the design of LLM-based multi-agent systems, but the direct evidence currently available is insufficient to establish these mechanisms as empirical properties of the domain. The constructs evaluated constitute plausible experimental hypotheses whose validation requires a dedicated experimental program.