Unwanted variability in child welfare decision-making: an empirical investigation

Assessing a parent’s ability to provide safe, nurturing care is critical yet difficult. Child welfare professionals must make such judgements, but they are complex and prone to noise, understood as unwanted variability. Noise arises when decisions rely on ambiguous information and discretion-laden criteria, a common feature in child welfare. In Study 1 (N = 49), professionals independently rated the overall quality of care in ten vignettes varying in concern severity, information completeness, and source. Inter-rater reliability was .50, with 24.7% of the variance due to systematic rater differences, suggesting varying thresholds for unsafe parenting. In Study 2 (N = 44), we tested whether averaging domain-specific evaluations could reduce noise. Although reliability improved modestly, the change was not significant. These results highlight the need for noise-reduction strategies to improve consistency in child welfare assessments. Because noise is measurable and predictably introduces error, we recommend integrating targeted noise-reduction measures.