Multimodal Fake News Detection Based on Large Language Models and Visual Models

With the rapid advancement of social media and online platforms, the dissemination of fake news has emerged as a significant threat to public opinion, public safety, and political stability, making fake news detection technology a critical area of global concern. This study proposes a multimodal feature extraction and fusion-based approach for fake news detection, aiming to enhance the efficiency of fake news classification by integrating features from both textual and visual modalities. First, we introduce a dual-LLaMA large language model architecture based on the LLMembed [1] framework, which significantly improves textual feature extraction. Subsequently, we incorporate three advanced image models—BLIP, CLIP, and ViT to achieve higher accuracy and efficiency in extracting image features. Finally, we propose a weighted feature fusion strategy that effectively combines information from both modalities by appropriately adjusting the weights of textual and visual features, ensuring each feature is fully utilized. Experimental results on several standard fake news detection datasets demonstrate the effectiveness of the proposed method.