Reproducible Expert Weight Elicitation via LLM Multi-Agent Simulation: A Best–Worst Method Decision Support Framework for AI-Driven E-Commerce Platform Evaluation

The pervasive integration of artificial intelligence across e-commerce ecosystems has fundamentally transformed the competitive landscape, rendering systematic and reproducible platform evaluation frameworks an operational necessity rather than an academic exercise. Conventional multi-criteria decision analysis approaches for e-commerce evaluation remain structurally constrained by their dependency on human expert panels, which introduce recruitment costs, cognitive biases, limited reproducibility, and the practical infeasibility of assembling genuinely multidisciplinary panels spanning e-commerce strategy, machine learning engineering, and financial technology simultaneously. This study proposes a novel decision support framework that integrates Large Language Model (LLM) multi-agent simulation with the Best–Worst Method (BWM) to derive reproducible priority weights for AI-driven e-commerce platform evaluation within a rigorous business intelligence architecture. Twelve domain-differentiated LLM agents—organized into three expertise groups representing e-commerce management, AI and machine learning technology, and digital payment systems—were instantiated with structured system prompts encoding professional domain knowledge and deployed across three independent simulation rounds to perform BWM pairwise comparisons across a comprehensive six-dimensional, 30-sub-criterion evaluation hierarchy. Inter-agent consensus was synthesized through geometric mean aggregation, with consistency verification conducted via BWM’s xi* indicator and inter-round stability assessed through coefficient of variation analysis. Results reveal that Transaction Security and Trust achieves the highest dimension-level weight (w = 0.248), followed by AI Recommendation Effectiveness (w = 0.213), with Personal Data Protection (G = 0.0750), Recommendation Accuracy (G = 0.0607), and Transaction Transparency (G = 0.0549) emerging as the three highest globally ranked sub-criteria. The aggregated consistency indicator xi* = 0.062 confirms logical coherence of the multi-agent judgment consensus, and all dimension weights exhibit CV values below 2.8%, demonstrating exceptional inter-round stability. Spearman rank correlations among the three domain-expertise groups exceed 0.92, confirming strong inter-group convergence. Sensitivity analysis under perturbations of ±10% and ±20% demonstrates that the top-five priority indicators are structurally stable. This study establishes LLM multi-agent BWM simulation as a methodologically rigorous, institutionally accessible, and computationally reproducible alternative to traditional expert elicitation for complex platform evaluation tasks.