Interrater Reliability of the Rorschach Performance Assessment System (R-PAS) in a Sample of Children in Child Protective Services

Abstract: To date, no interrater reliability study of the Rorschach Performance Assessment System (R-PAS) has focused exclusively on school-age children. Because child protocols are quantitatively and qualitatively different from adult protocols, interrater reliability could differ across age groups. This study sought to examine the interrater reliability of the R-PAS variables in a clinical sample of school-age children. The sample comprised the files of 60 children 6–12 years old with substantiated maltreatment reports (most in out-of-home care). All of the children received a psychological/neuropsychological assessment as part of the regular Child Protection Services (CPS) trajectory. The majority of children had a neurodevelopmental disorder, the most common being attention-deficit/hyperactivity disorder (ADHD) and developmental language disorder (DLD). R-PAS protocols were coded by two trained coders. Interrater agreement proved good to excellent (intraclass correlation over .59) on the vast majority of R-PAS variables (92%). Specifically, reliability proved poor for 2% of the variables (one variable), fair for 7% (four variables), good for 23% (14 variables), and excellent for 68% (41 variables). Mean ICC was .80. The only score with poor interrater reliability was CBlend. The mean ICC was comparable for children with and without DLD. Overall, the results indicate that the R-PAS possesses good-to-excellent interrater reliability even with young and vulnerable children. Study limitations include a restricted clinical sample and the highly intensive training of the two coders.