GPT-4o reads the mind in the eyes

Humans possess a sophisticated ability to read the mind in the eyes of other people. Here we tested whether this ability is also present in GPT-4o, a multimodal LLM. Using two versions of a widely used theory of mind test, the Reading the Mind in Eyes Test and the Multiracial Reading the Mind in the Eyes Test, we found that GPT-4o outperformed humans in extracting mentalistic information from upright faces but underperformed humans when faces were inverted. GPT-4o errors were not random but revealed a highly consistent, yet incorrect, processing of mental-state information across trials, with an orientation-dependent error structure that qualitatively differed from that of humans for inverted faces but not for upright faces. These findings highlight how advanced mental state inference abilities coexist in GPT-4o alongside substantial differences in information processing compared to humans. • GPT-4o surpasses humans in reading mental states from the eye-region. • Face inversion severely reduces GPT-4o’s mindreading. • Unlike human errors, GPT-4o’ errors are not random. • Error analysis reveals GPT-4o’s highly structured error space.