The robust beauty of confidence filtering

Confidence is an imperfect guide to accuracy, but a group need not use it to identify its best member or assign a precise weight to every answer. Confidence can instead determine whether an answer contributes at all. This article examines confidence filtering: responses below a threshold abstain, and the remaining responses are combined by an equally weighted majority vote. Three studies applied specified decision rules to existing behavioral judgments. In binary-choice inference, filtering improved the exact expected accuracy of groups of 30 respondents at every tested threshold from 10 to 70 on a 0–100 confidence scale. The largest improvement was 2.73 percentage points, and a fixed threshold also benefited larger groups. A second study extended the analysis to counterfactual transmission chains constructed from physicians’ diagnostic judgments. The modified rule allowed low-confidence disagreement to suspend an inherited answer without asserting an alternative. This reduced overall accuracy but partially mitigated the deterioration found in cases where the original majority was wrong. The improvement depended on resolving final undecided states at chance; it did not reflect an increase in committed correct answers. A third study compared filtering with the surprisingly popular method. In general knowledge, every integer threshold above the confidence-scale minimum outperformed both ordinary majority voting and the surprisingly popular benchmark. The highest accuracy was 61%, compared with 51% and 50%, respectively. NFL forecasting provided a contrasting result: filtering did not improve ordinary majority voting, and strict thresholds reduced accuracy. All items remained in the analyses, with unresolved outcomes receiving half credit. Confidence filtering therefore worked across broad threshold ranges in some settings, rather than providing a universally superior rule. Its value lies in using confidence to decide which answers should contribute and, in sequential transmission, when an inherited answer should cease to be treated as settled.