Building on Shaky Foundations: A Critical Review of Criterion Validity Evidence in Personnel Selection

ABSTRACT We reviewed the most recent meta‐analytic literature on the criterion validity of predictors of job performance used in personnel selection. The overreliance on concurrent incumbent studies leads to meta‐analytic estimates based on data collected in conditions with non‐representative stakes and an inability to estimate and correct for range restriction based on credible information, obstructing the estimation of operational validity in actual selection contexts. Therefore, we argue that we do not need more meta‐analyses. First, we need primary studies that actually provide estimates of operational criterion validity. On a positive note, we observed increasing attention in meta‐analyses to heterogeneity of results, publication bias and open science. There is room for improvement in terms of attention to the quality and operationalization of predictor and performance measures, and reflection on the quality of the evidence. Given these issues, our stance is that the current body of research does not allow drawing firm conclusions about the criterion validity of predictors of job performance in actual, operational selection settings. To move forward, we provide recommendations for reviewers, editors, and authors of primary studies and meta‐analyses on predictors used in personnel selection.