Assessing bias in informant ratings: Development of an Atypical Change Index for the Iowa Scales of Personality Change

Objective: Neuropsychological assessments should employ scientifically supported instruments to detect bias. To our knowledge, there are no validated measures for assessing bias in informant ratings of personality changes following brain injuries. We report the development of such a measure for the Iowa Scales of Personality Change (ISPC). Method: ISPC personality change ratings were obtained from informants for: (1) 48 Research Registry patients with chronic, focal, ventromedial prefrontal cortex lesions (vmPFC group), and (2) 226 individuals undergoing independent neuropsychological evaluations in the context of litigation (INE group). An “Atypical Change Index” was developed using vmPFC group ISPC data, comprising scales for which extreme changes were rare among vmPFC patients (≤2.5%). We hypothesized that the INE group would have higher Atypical Change Index scores compared to the vmPFC group. To avoid circularity, we used repeated cross-validations to separate the vmPFC group data into “train” sets for identifying scales for the Atypical Change Index, and “test” sets for comparing the Atypical Change Index across vmPFC and INE groups. Results: The INE group had statistically significantly higher Atypical Change Index scores than the vmPFC group (p < .001). One vmPFC patient (2.1% of the group) had an Atypical Change Index score > 1, compared to 64 INE patients (28.3% of the group) with scores > 1. Conclusions: Our findings provide initial support for the potential of the Atypical Change Index to detect bias in informant ratings on the ISPC. Further investigation is required to evaluate the validity of this index.