A real-time keypoint-based graph convolutional network for suspicious activity recognition in surveillance systems

The increasing demand for real-time video surveillance in public environments has exposed major limitations in existing systems, particularly their inability to handle visual challenges such as occlusions, low-light conditions, and dynamic backgrounds. These factors often degrade the reliability of traditional CNN- or LSTM-based activity recognition models. To address these shortcomings, we propose KPARRTS (KeyPoint-based Activity Recognition using Real-Time Spatio-temporal Graphs), a lightweight and robust framework designed to operate effectively under complex conditions. Unlike conventional methods that separate spatial and temporal processing stages—often leading to poor performance under occlusion, low-light, and dynamic background conditions— KPARRTS introduces a unified, lightweight GCN-based spatiotemporal framework that enhances detection accuracy and reduces latency, outperforming existing baselines in challenging surveillance scenarios. The system integrates MediaPipe-based keypoint extraction with a two-layer Graph Convolutional Network (GCN) that models human motion across frames. A hybrid training dataset comprising COCO, MPII, and real-world surveillance footage improves generalization across diverse conditions. Experimental results show that KPARRTS achieves a mean Average Precision of 74.5%, an F1-score of 81.2%, and an inference time of 39 ms per frame, using under 2.1 GB of GPU memory. Compared to OpenPose-LSTM and CNN-LSTM baselines, KPARRTS achieves 12.4% and 9.1% higher mAP, respectively, while reducing computational load by up to 50%. These findings demonstrate the model’s practical value for real-time surveillance applications, including threat detection and public safety monitoring.