The next frontier in mindreading? Assessing generative artificial intelligence (GAI)'s social-cognitive capabilities using dynamic audiovisual stimuli

The integration of Generative Artificial Intelligence (GAI) into human social contexts has raised fundamental questions about machines’ capacity to understand and respond to complex emotional and social dynamics. While recent studies have demonstrated GAI’s promising capabilities in processing static emotional content, the frontier of dynamic social cognition – where multiple modalities converge to create naturalistic social scenarios – remained largely unexplored. This study advances our understanding by examining the social-cognitive capabilities of Google’s Gemini 1.5 Pro model through its performance on the Movie for the Assessment of Social Cognition (MASC), a sophisticated instrument designed to evaluate mentalization abilities using dynamic audiovisual stimuli. We compared the model’s performance to a human normative sample (N = 1,230) across varying temperature settings (a parameter controlling the level of randomness in the AI's output, where lower values lead to more deterministic responses and higher values increase variability; set at 0, 0.5, and 1). Results revealed that Gemini 1.5 Pro consistently performed above chance across all conditions (all corrected p s < .001, Cohen's h range = 1.17-1.41) and significantly outperformed the human sample mean ( Z = 2.24, p = .025; Glass's Δ = 0.92, 95% CI [0.11, 1.72]; Hedges' g = 0.92, 95% CI [0.12, 1.72]). Analysis of error patterns revealed a distribution between hyper-mentalizing (41.0%; over-attribution of mental states), hypo-mentalizing (46.2%; under-attribution of mental states), and non-mentalizing (12.8%; failure to recognize mental states) errors. These findings extend our understanding of artificial social cognition to complex multimodal processing while raising important questions about the nature of machine-based social understanding. The implications span theoretical considerations in artificial Theory of Mind to practical applications in mental health care and social skills training, though careful consideration is warranted regarding the fundamental differences between human and artificial social cognitive processing. • Generative artificial intelligence (GAI) surpasses human average performance in standardized mindreading assessment • Dynamic audiovisual stimuli reveal GAI's sophisticated social-cognitive processing capabilities • Our findings establish new pathways for GAI applications in mental health assessment and intervention