TamperNet: A Spatio-Temporal Deep Learning Framework for Video Manipulation Detection

Abstract The modern tools used for video editing and compression make video tampering detection extremely difficult for digital forensic investigators, as it can often hide visual and temporal evidence or artifacts of tampering. In this work, we propose TamperNet, a hybrid spatio-temporal framework to detect various video manipulations that are common in both inter-frame and intra-frame domains such as frame duplication, frame deletion, cloning, splicing, and inpainting. The key design contribution is the spatio-temporal feature fusion by concatenation, a temporal representation module using LSTM, a spatio-temporal descriptor using ResNet-50, and a spatio-temporal anomaly scoring strategy based on temporal prediction error and autoencoder reconstruction error. Results on the Video Forgery Dataset (VFD) indicate an accuracy of 93.3%, a precision of 91.4%, a recall of 91.3%, an F1-score of 89.9%, and an ROC-AUC of around 0.75. These findings show that TamperNet works on the tested benchmark set, but further cross-dataset testing and compression-specific testing are required for assessing the potential for generalization to the real world, given the moderate ROC-AUC and small size of the tested benchmark set. Other future research will include validation on larger benchmark sets, making the framework more robust in various compression and acquisition scenarios, and the extension of the model to enable localization of tampering.