Subtracting Self-Selection Bias from Scientific Inquiry: A Computational Exploration

Abstract In this paper, I examine the impact of self-selection bias on expert consensus in fields driven by constructed arguments, such as philosophy. Through an agent-based model, I explore how pre-existing beliefs influence researchers' decisions to enter specific fields and how these dynamics contribute to distorted epistemic outcomes, particularly when the questions investigated are complex and argument construction plays a central role. The model reveals that self-selection can significantly and stably skew expert consensus, reinforcing initial biases rather than reflecting the evidence. To address this, I propose and test several strategies that scientific reviewers relying on expert testimony can employ to subtract this bias when relying on expert opinions. The results suggest that strategies accounting for the direction and persistence of belief changes among experts are particularly effective in improving the accuracy of reviewers' conclusions.