适应视觉语言模型实现通用深度伪造检测

With the rapid development of generative networks, the authenticity of information is facing unprecedented challenges. The misuse of this technology could lead to a proliferation of fake images, severely undermining the credibility of visual content. If such fake images go undetected and are used in scientific research, journalism, or law, they could mislead public perception and challenge scientific conclusions. Therefore, developing efficient image authenticity detection techniques has become an urgent need. Current research focuses on using large-scale datasets of generated images to train detectors. However, these data-intensive methods often incur high computational costs and exhibit limited generalization capabilities when dealing with unseen generated images. This paper proposes a few-shot learning approach for detecting AI-generated images. To capture high-level semantic information, the method leverages visual features extracted by a pre-trained CLIP model. Given the limitations of existing models in learning representations of generated images, a novel adaptive forgery detection algorithm is designed. This approach introduces two key modifications to the CLIP model to enhance its ability to learn broader forgery representations. First, unlike previous studies that only utilize the visual encoder, both the image and text encoders of the CLIP model are employed here to exploit its capabilities fully. Second, a simple and lightweight prompt optimization strategy is adopted to make the CLIP model better suited for forgery representations, significantly improving performance with only a small amount of training data. Furthermore, considering that existing studies typically focus on training with images from a single generative model, an alternative method is proposed that does not require additional training. By incorporating knowledge from more advanced generative models, the generalization ability of the model is significantly enhanced. Experimental results show that, through these two design modifications, the proposed method achieves an average detection accuracy that surpasses the current state-of-the-art by 10.82% across multiple public datasets.