Selection Regimes and Selection Errors

How can the selection of innovation projects be designed to reduce false positives and false negatives? Prior research has provided theoretical insights into organizing to reduce errors, yet we know little about how organizations adapt selection over time and the effects of this on selection outcomes. Drawing from qualitative data from 126 interviews conducted over several years, we explore how an accelerator evolved through three selection regimes for high-stakes funding decisions, focusing on the organizational changes and their underlying reasons. We then analyze quantitative data from all 3,580 submissions they received, assessing false positives and false negatives across these regimes. Our findings reveal a persistent occurrence of both types of errors, with relatively small differences across the regimes despite deliberate efforts to enhance the process. In the final regime, which increased submission quality by emphasizing applicant track record and adding additional layers of screening, evaluators surprisingly became more prone to making selection errors. This finding stands net of accounting for (1) differences in the pool of submissions, (2) differences in treatment effects through training and resources provided, (3) learning, and (4) market evolution. By combining qualitative and quantitative data, we explain this through two mechanisms: (1) mean reversion in combination with increased emphasis on applicant track record and (2) within-type adverse selection enabled by a more stringent selection process. The study reveals that evolving an organization’s selection regime may require adjustments across multiple aspects, resulting in unintended consequences. Supplemental Material: The online appendix is available at https://doi.org/10.1287/orsc.2023.17482 .