Rethinking Standards in Child Maltreatment Risk Assessment: Are “Fair” and “Moderate” Reliability Enough When Child Safety Is at Stake?

Child maltreatment constitutes a major public health concern in Australia and internationally. Despite its widespread prevalence and consequences, empirical guidance on how best to assess maltreatment risk within families is significantly lacking. This study aimed to assess the inter-rater reliability of two Structured Professional Judgement (SPJ) tools designed for the child maltreatment context: the Child Abuse Risk and Resilience Assessment (CARRA) and the Child Protection Removal Assessment (Chi-PRA). The sample included 40 parent–child dyads (25 children, 40 parents), randomly selected to represent 10% of a larger cohort. Cases were drawn from court-ordered psychological assessments of Family Division matters completed by the Children’s Court Clinic in Victoria, Australia. Through retrospective file review, three researchers independently coded each case using the CARRA and Chi-PRA. Inter-rater reliability was assessed using percent agreement and Fleiss’ kappa (κ), calculated for individual items and summary risk judgements. For the CARRA, κ values ranged from –0.01 to 0.69 across individual items and 0.35 to 0.45 for summary risk ratings, indicating poor-to-moderate agreement. In a secondary item-level analysis collapsing “Yes” with “Partial” responses and “No” with “Missing,” modest gains were observed, though many items remained below acceptable thresholds. The Chi-PRA demonstrated stronger agreement at both the item-level (κ = 0.31–0.78), and summary rating (κ = 0.77). These findings contribute to the limited evidence base for SPJ measures in child maltreatment and provide direction for further evaluation of their reliability, validity, and clinical utility in this context.