Enhancing surveillance anomaly detection with keyframes and explainable inception model

The proliferation of surveillance systems in the 21st century has become critical in mitigating rising crime rates and preserving life and property. However, conventional surveillance methods struggle with detecting abnormal human activities in complex environments characterized by varying lighting conditions and diverse appearances of individuals. To address these challenges, we propose advanced computer vision techniques, particularly supervised anomaly detection, to enhance intelligent surveillance systems. This research introduces SilentFrm to optimize anomaly detection by analyzing frame differences and setting thresholds based on calculated areas. SilentFrm streamlines the keyframe selection process, enhancing efficiency and reducing redundancy in anomaly detection tasks. Additionally, we present XAI-Inv3, an innovative architecture that integrates interpretability techniques such as Grad-CAM and guided backpropagation into the InceptionV3 model. These methodologies exhibit high accuracy and provide interpretable insights, thereby enhancing situational awareness and security. Rigorous experimentation showcases SilentFrm and XAI-Inv3′s superior detection accuracy and interpretability compared to state-of-the-art methods. By leveraging insights from weak supervision and semi-supervised learning, our approach aims to develop robust anomaly detection models capable of discerning anomalies with precision and efficiency in dynamic surveillance environments. Integrating XAI-Inv3 with the keyframes approach facilitates real-time anomaly detection and ensures scalability and feasibility across large-scale surveillance deployments. The proposed model achieved an average accuracy of 99 % on Hockey Fight, Violent Flow and Real-Life Violence Situation datasets for violence detection and 92 % for violence recognition using UCF-Crime and Shanghai Tech datasets. Proposed contributions underscore the importance of innovative methodologies in advancing the field of surveillance video analysis, ultimately enhancing public safety and crime prevention efforts.