Correlation limits the reliability of ensemble decision making with large language models

Abstract Multi-agent systems increasingly combine the votes of several Large Language Models (LLMs), assuming that an ensemble is more reliable than its members. The classical warrant, the Condorcet Jury Theorem, requires independent voters, yet models that share pre-training data, architectures, alignment, or prompts tend to fail on the same questions. This paper argues that the correlation between agents’ errors, which design choices influence but do not fix, is the decisive and most overlooked quantity in LLM ensemble decision-making. Within a model of exchangeable binary votes, we adapt known results on correlated juries. The effective number of independent voters, a variance-based effective sample size, is capped by the reciprocal of the correlation: a hundred agents correlated at 0.3 have the variance of about three independent ones. Majority accuracy plateaus below certainty and can fall below that of one agent. Adding samples from one model at a fixed temperature cannot raise that plateau, whereas different base models can; whether persona prompts can is empirical. We extend the effective-size result to unequal correlations and competences, give a moment estimator of the correlation with bootstrap intervals, and apply it to real models on MMLU: temperature samples and personas of one model are correlated at 0.85, five models from different developers at 0.38. We argue that semantic aggregation does not automatically escape correlated inputs, and propose a six-item reporting checklist for deployed ensembles. Among language models, consensus is not by itself evidence of correctness; among correlated agents, it may reflect redundancy.