SSTAR: Skeleton-Based Spatio-Temporal Action Recognition for Intelligent Video Surveillance and Suicide Prevention in Metro Stations

Recent advances in computer vision greatly improved human behavior analysis in videos, particularly for developing smart surveillance systems that can detect and understand critical human actions in real-world situations. An important potential public health and public safety use for these systems is to automatically identify pre-suicidal behaviors, allowing for quick interventions to prevent these tragedies. In this paper, we propose SSTAR, a novel Skeleton-based Spatio-Temporal Action Recognition model. SSTAR incorporates a 1D-Swin Transformer to capitalize on its hierarchical and multi-scale representation capabilities for efficient spatial feature extraction. For temporal modeling, we employ an sLSTM block to effectively capture long-term dependencies and dynamic motion patterns. We also propose a new real- world dataset capturing passenger actions associated with pre-suicidal behaviors, named ARMM. Extensive experiments on two public benchmark datasets, JHMDB and NW-UCLA, demonstrate the state-of-the-art performance as well as the high computational efficiency of our proposed approach. Moreover, we demonstrate the effectiveness of SSTAR in identifying action patterns for suicide prevention on our new ARMM dataset, achieving 91.34% accuracy and outperforming other skeleton-based action recognition methods. The code and dataset are available at https://github.com/SafwenNaimi/SSTAR.