Do Multi-Agent LLM Systems Actually Help? A Controlled Evaluation of Single-Agent, LLM-Orchestrated, and Deterministically Orchestrated Architectures

This preprint presents a controlled evaluation of the three LLM-based transactional agent architectures: a single general-purpose agent agent, an LLM-orchestrated multi-agent system, and a deterministically orchestrated multi-agent system. All three systems used the same models, tools, business rules, synthetic data, benchmark tasks, and scoring procedure. The study evaluated 30 transactional tasks across three repetitions per system, producing 270 measured runs. Under the frozen trace-based scorer, the single-agent system achieved 94.44% task success, the LLM-orchestrated multi-agent system achieved 74.44%, and the deterministically orchestrated multi-agent system achieved 93.33%. The LLM-orchestrated architecture also required substantially more model calls, tokens, and latency. Trace-level analysis showed that several failures in the LLM-orchestrated system resulted from inter-agent coordination problems, including loss or corruption of critical state such as authoritative quote identifiers. Deterministic orchestration recovered most of this lost reliability by valdiating prerequisites and retrieving tool outputs before critical state-changing actions. However, the deterministic architecture also showed reduced flexibility on some conditional instructions. The results suggest that increasing the number of agents does not inherently improve reliability. In this controlled transactional setting, explicit state management and deterministic control of critical workflow transitions were more important than multi-agent decomposition alone.