Temporal attention–based hybrid neural network model for human behavior analysis in video surveillance

Recently, the occurrence of criminal activities has risen significantly. Based on the reports of the National Crime Records Bureau of India, the crime rates in the last two decades have increased from 215.5 to 258.1 with the population percentage of 13.3% and 8.12%, respectively. The criminal activities include kidnapping, theft, assault, robbery, and crimes against women. Most of the attacks took place in public areas that are more complex to track and prevent. As video surveillance has become a part of all public and private places, it can be used for monitoring the behavior of individuals and preventing any abnormal activity by applying proper mechanisms. Anomaly detection is the process of identifying any irrelevant activities that deviate from normal activities and reporting them through an autonomous process. Anomaly detection systems analyze the behavior of people by monitoring them, recognizing their activities, analyzing their activity patterns, and making an alert when an abnormal activity pattern is detected. However, monitoring such videos using manpower and analyzing them in crowded places is impractical. So, an automatic system for monitoring and analyzing video data is required to enable abnormal activity detection. These systems will be helpful in various applications such as crime mitigation, patient monitoring systems, and security surveillance for public meetings. An anomaly detection system with a temporal attention–based hybrid model that combines modified long short-term memory and convolutional neural network (ABLSTM_CNN) has been deployed, and the performance metrics such as accuracy and loss function are presented. For deploying the model and evaluating the performance, two datasets such as the human activity recognition (HAR) dataset and the UCF_crime dataset have been used. This system achieves an accuracy of 92.28% with a loss of 0.5 to 0.18 on HAR dataset and achieves 0.83 AUC on UCF_crime dataset. The model is evaluated with other metrics such as precision, recall, F1 score, and support.