Human–AI Divergence in Ego-Centric Action Recognition Under Spatial and Spatiotemporal Manipulations

Humans consistently outperform state-of-the-art AI models in action recognition, particularly in challenging real-world conditions involving low resolution, occlusion, and visual clutter.e those utilised by humans Understanding the sources of this performance gap is essential for developing more robust and human-aligned models. In this paper, we present a large-scale human AI comparative study of egocentric action recognition using human-defined Minimal Identifiable Recognition Crops (MIRCs), defined as the smallest spatial or spatiotemporal regions sufficient for reliable human recognition. We used our previously introduced, Epic-ReduAct, a systematically spatially reduced and temporally scrambled dataset derived from 36 EPIC-KITCHENS videos, spanning multiple spatial reduction levels and temporal conditions. Recognition performance is evaluated using over 3,000 human participants, Side4Video, and Qwen 3.5 9B models. Our analysis combines quantitative metrics, Average Reduction Rate and Recognition Gap, with qualitative analyses of spatial (high-, mid-, and low-level visual features) and spatiotemporal factors, including a categorisation of actions into Low Temporal Actions (LTA) and High Temporal Actions (HTA). Results show that human performance exhibits sharp declines when transitioning from MIRCs to sub-MIRCs, reflecting a strong reliance on sparse, semantically critical cues such as hand object interactions. In contrast, the model degrades more gradually and often relies on contextual and mid- to low-level features, sometimes even exhibiting increased confidence under spatial reduction. Temporally, humans remain robust to scrambling when key spatial cues are preserved, whereas the model often shows insensitivity to temporal disruption, revealing class-dependent temporal sensitivities. Overall, our findings demonstrate that strong benchmark performance can obscure fundamental human AI misalignment and highlight limitations in current spatial and spatiotemporal representations, suggesting concrete directions for more efficient and human-aligned action recognition systems.

Authors

Institutions

Publication Details

Journal
International Journal of Computer Vision
Published
2026-11-19
DOI
https://doi.org/10.1007/s11263-026-03077-7
Primary Topic
Human Pose and Action Recognition
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Human–AI Divergence in Ego-Centric Action Recognition Under Spatial and Spatiotemporal Manipulations

Quoc C. Vuong, Frank Guérin, Anya C. Hurlbert, Sadegh Rahmaniboldaji et al.
3 links
International Journal of Computer Vision
Human Pose and Action Recognition
article

Human–AI Divergence in Ego-Centric Action Recognition Under Spatial and Spatiotemporal Manipulations

Quoc C. Vuong, Frank Guérin, Anya C. Hurlbert, Sadegh Rahmaniboldaji, Filip Rybansky, Andrew Gilbert
article en

Abstract

Humans consistently outperform state-of-the-art AI models in action recognition, particularly in challenging real-world conditions involving low resolution, occlusion, and visual clutter.e those utilised by humans Understanding the sources of this performance gap is essential for developing more robust and human-aligned models. In this paper, we present a large-scale human AI comparative study of egocentric action recognition using human-defined Minimal Identifiable Recognition Crops (MIRCs), defined as the smallest spatial or spatiotemporal regions sufficient for reliable human recognition. We used our previously introduced, Epic-ReduAct, a systematically spatially reduced and temporally scrambled dataset derived from 36 EPIC-KITCHENS videos, spanning multiple spatial reduction levels and temporal conditions. Recognition performance is evaluated using over 3,000 human participants, Side4Video, and Qwen 3.5 9B models. Our analysis combines quantitative metrics, Average Reduction Rate and Recognition Gap, with qualitative analyses of spatial (high-, mid-, and low-level visual features) and spatiotemporal factors, including a categorisation of actions into Low Temporal Actions (LTA) and High Temporal Actions (HTA). Results show that human performance exhibits sharp declines when transitioning from MIRCs to sub-MIRCs, reflecting a strong reliance on sparse, semantically critical cues such as hand object interactions. In contrast, the model degrades more gradually and often relies on contextual and mid- to low-level features, sometimes even exhibiting increased confidence under spatial reduction. Temporally, humans remain robust to scrambling when key spatial cues are preserved, whereas the model often shows insensitivity to temporal disruption, revealing class-dependent temporal sensitivities. Overall, our findings demonstrate that strong benchmark performance can obscure fundamental human AI misalignment and highlight limitations in current spatial and spatiotemporal representations, suggesting concrete directions for more efficient and human-aligned action recognition systems.

International Journal of Computer VisionVol. 134(12)
University of Surrey (GB), Newcastle University (GB)
Leverhulme Trust
Human Pose and Action Recognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon