Ignorance Is Bliss: Evaluating the Impact of Priors, Correlations, and Base Rates in Universal Screening Cost–Accuracy Analysis

Universal screening in schools is a costly endeavor, but the use of cost–accuracy ratios may augment its positive impacts on school-based prevention. In addition, educational researchers have more recently begun to adopt Bayesian methods of calculating screener accuracy, such as post-test probabilities or fully Bayesian statistical models. However, there is no research evaluating how screener costs intersect with the properties of Bayesian statistical modeling. In this study, we conducted a simulation experiment of how priors within Bayesian logistic regression influenced the estimation of screening costs as an interaction between both the true correlation of a screener (e.g., from the normative sample) and base rates of actual risk. We found that the cost of falsely predicting a student was at risk was substantially higher with a lower base rate of .2, regardless of the strength of the true correlation between the screener and criterion outcome. Our results also demonstrated that costs were most sensitive to informative priors, even when the priors were designed to be very similar to the true simulation values. Results demonstrated the practical application of Bayesian analyses for evaluating how prior assumptions can influence screening cost estimation, and we provide practical guidance for how these methods can benefit practitioners and policymakers.