SFTA-Net: a self-supervised approach to detect copy-move and splicing forgery to leverage triplet loss, auxiliary loss, and spatial attention

Image forgery is an increasing threat, fueling misinformation and potentially impacting legal decisions and everyday life. Detecting forged media, including images and videos, is crucial for preserving trust and integrity across various platforms. Common forgery techniques like copy-move and splicing require robust detection methods to identify tampered areas without explicit guidance. The previously proposed studies focused on a single type of forgery detection utilizing block-based and key-point feature selection-based classical machine learning (ML) approaches. Furthermore, applied deep learning (DL) methods only focus on deep feature extraction without considering the focus on tampered regions detection or any domain-specific loss. Therefore, this study addresses the aforementioned challenges by proposing a lightweight DL approach, a self-supervised, triplet and auxiliary losses-based forgery detection network (SFTA-Net), featuring a self-guidance mechanism for detecting tampered regions with a commutative loss within images. The SFTA-Net method is proposed to classify forged and original photos belonging to copy-move and splicing forgeries. To effectively analyze the added components in the proposed model, three experiments were conducted, one with a self-guided (SG) head-based convolutional neural network (CNN), a second with SG-head and auxiliary loss, and a third one with SG-head auxiliary loss and triplet losses-based CNN. For experimentation, CASIA 1.0 and CASIA 2.0 datasets were used with 80-10-10% train-validation and test ratios. The testing results achieved on CASIA 1.0 were 95% accuracy and 97% accuracy on the CASIA 2.0 dataset. To prove the approach's robustness and generalization, the CASIA 2.0-trained weights were used to test on the MICC-FC2000 dataset and yielded limited results. To improve the results, fine-tuning was performed on CASIA 2.0 weights utilizing the MICC-FC2000 dataset which achieved 98% accurate results. Our findings demonstrate that the SFTA-Net surpasses the baseline ResNet18 model and previous state-of-the-art (SOTA) methods. Overall, our SG approach offers a promising solution for detecting forged images across diverse real-world scenarios, contributing to the mitigation of image forgery and preservation of trust in digital content.