Optimizing Group Activity Recognition With Actor Relation Graphs and GCN-LSTM Architectures

The challenge of understanding and recognizing group activities through human behavior and interactions is a prominent issue in the realm of computer vision research. This area boasts a wide range of applications, including security surveillance, healthcare monitoring, and human-computer interaction systems. However, accurately deciphering complex group dynamics continues to pose significant difficulties. Traditional video-based activity recognition methods often grapple with persistent obstacles such as environmental noise, background clutter, and the inter-class similarities inherent in activity patterns. Recent advancements in this field have largely focused on spatial feature extraction, which alone proves inadequate for thorough group activity analysis. In response to these limitations, we propose a novel deep learning framework that effectively captures both temporal and spatial characteristics of group interactions. Our architecture employs a Convolutional Neural Network (CNN) with Inception-V3 as the foundational model for initial feature extraction. This is complemented by the development of an Actor Relation Graph (ARG) utilizing Zero Normalized Cross Correlation (ZNCC), which adeptly illustrates both appearance-based and positional relationships among participants. By integrating the ARG with a hybrid model that combines Graph Convolutional Network (GCN), Long Short-Term Memory (LSTM), and Attention mechanisms, our approach significantly enhances the extraction of spatial and relational features compared to conventional techniques. Experimental evaluations conducted on two benchmark datasets—the Collective Activity Dataset (CAD) and the Volleyball dataset—demonstrate the efficacy of our framework. Our proposed model achieves state-of-the-art performance, attaining prediction accuracies of 94.32% and 94.47% on the CAD and Volleyball datasets, respectively, thereby surpassing existing methodologies in the realm of group activity recognition.