A multi-dimensional evaluation method for large language model-powered multi-agent systems

The multi-agent system driven by large models has great potential in enhancing the level of artificial intelligence, providing innovative solutions for military intelligence. However, the degree of autonomy of current large model driven multi-agent systems in independently completing task objectives is greatly affected by task complexity, and the consistency between system processing results and initial objectives is poor. It is necessary to evaluate and analyze the autonomy and consistency of large model driven multi-agent systems. Previous studies have not yet comprehensively evaluated the autonomy and consistency levels of large model multi-agent systems. This article proposes a multidimensional evaluation method that can analyze and extract the autonomy and consistency of the overall architecture of multi-agent systems driven by large models, and obtain the overall performance evaluation results and specific improvement methods of the system. Through experimental analysis of 7 selected systems, the feasibility of multidimensional evaluation methods in practical applications has been verified.