Deep hashing framework for proactively deepfake retrieval and detection

In contemporary times, the internet is inundated with manipulated images known as Deepfakes. Additionally, copyright infringement is rampant, as users’ real images uploaded to the internet are often used by forgers to generate fake images for malicious purposes, posing significant security and privacy concerns. While current Deepfake detection methods are often utilized only after the fake images have already caused harm. To address this issue, we propose a proactive and explainable Deepfake detection framework, named HashVAE, by combining image retrieval with Deepfake detection. Specifically, first, a hash feature extractor is proposed, utilizing a multi-branch residual mechanism at varying scales to fully explore hierarchical features for foreground and background representation. To enable the framework with Deepfake detection capabilities, a Deepfake detection block (DDB) is attached to the foreground hash feature. In addition, a quadruplet loss is designed to strengthen the relation between query hash code space and database hash code space. To achieve Deepfake retrieval and detection, a two-step forgery retrieval strategy which utilize both foreground and background information to detect suspicious images is designed. Notably, our proposed solution reduces the relationship between iterative updates of generative techniques and classifier, making it independent of current artifact mining strategies for forgery detection.