A multi-site psychometric evaluation of the Reading the Mind in the Eyes Test – Revised

The Reading the Mind in the Eyes Test, Revised (RMET-R) is a widely used measure that purports to assess theory of mind (ToM). However, the psychometric properties of the RMET-R have been called into question. To examine a wider array of the RMET-R's psychometric properties, as compared to prior studies, we recruited undergraduate students ( N = 640; 65.31 % women; M age = 20.33; SD = 4.30) from three public universities. Participants completed the RMET-R, a novel word choice familiarity task, a novel task rating each response option's description of the picture, along with emotional valence and arousal, and two self-report cognitive empathy measures. Results indicated a small positive correlation with one self-report measure of cognitive empathy, and acceptable internal consistency. Performance was related to prior familiarity of words used on the test, and a quarter of the trials had a foil word rated similarly as the target word in describing picture. Item-level analysis found that valence ratings correlated significantly with accuracy, even on the least accurate items. Results corroborate and expand published concerns about the RMET-R's psychometric properties. We propose changes to the measure, including updated stimuli and structural improvements. • Internal consistency was in marginally acceptable range. • Accuracy was correlated with one of two self-report cognitive empathy measures. • Accuracy was correlated with ratings of familiarity with words used in task. • Quarter of trials had foil word rated similarly as target in describing picture. • Valence ratings were associated with accuracy, even on the least accurate items.