Violence Detection in Video Using Statistical Features of the Optical Flow and <scp>2D</scp> Convolutional Neural Network

ABSTRACT The rapid growth of video data has resulted in an increasing need for surveillance and violence detection systems. Although such events occur less frequently than normal activities, developing automated video surveillance systems for violence detection has become essential to minimize labor and time waste. Detecting violent activity in videos is a challenging task due to the variability and diversity of violent behavior, which can involve a wide range of actions, motions, and interactions between people and objects. Currently, researchers employ deep learning models to detect violent behaviors. In fact, a large number of deep learning approaches are based on extracting spatio‐temporal information from a video by exploiting a 3D Convolutional Neural Network (CNN). Despite their success, these techniques require a lot more parameters than 2D CNNs and have high computational complexity. Therefore, we focus on exploiting a 2D CNN to encode spatio‐temporal information. Actually, statistical features of the optical flow changes are used to give this ability to a 2D CNN. These features are designed to make attention to regions of a video clip with much more motion. Accordingly, the optical flow of an input video is calculated. To determine meaningful changes in the optical flow, the optical flow magnitude of a current frame is compared with its predecessor. After that, statistical features of these changes are extracted to summarize a video clip to a 2D template, which feeds a 2D CNN. Experimental results on four benchmark datasets observe that the suggested strategy outperforms baseline ones. In particular, we make a better estimation of the spatio‐temporal features in a video by shortening a video clip into a 2D template.