Unsupervised Video Anomaly Detection Using Video Vision Transformer and Adversarial Training

Surveillance cameras have been recently introduced in various locations to maintain public safety. However, it is tedious for security personnel to continue observing videos obtained by surveillance cameras because abnormal events rarely occur. Consequently, this study aims to develop an unsupervised anomaly detection method for surveillance videos based on the prediction of future frame and optical flow magnitude using adversarial networks. Our database consists of two public datasets for anomaly detection: UCSD Pedestrian 2 (UCSD Ped2) and ShanghaiTech Campus. The proposed network consists of a generator and a U-Net-based discriminator. The generator has an encoder based on a video vision transformer and two decoders that independently predict the future frame and optical flow magnitude using feature maps extracted from the encoder at different resolutions. The U-Net-based discriminator identifies at the pixel level whether the image is a “real” or “false” frame predicted by the generator. The generator and U-Net-based discriminator in the proposed network are trained through adversarial training. Finally, anomaly detection is performed based on element-wise subtraction between the predicted future frame and the optical flow magnitude and their corresponding ground truth frames. The areas under the receiver operating characteristic curve (AUC) with the proposed network were 97.5% for UCSD Ped2 and 85.8% for ShanghaiTech Campus. The AUC for the proposed network was also greater than those of well-known anomaly detection methods.