A Multi-Modal Rumor Detection Method Based on Topic Features and Multi-Level Fusion

Rumors can have a negative impact on social life, and multimodal rumors, which combine text and images, are more misleading and spread more widely than text-only rumors. Therefore, detecting multimodal rumors is particularly important. Existing multimodal rumor detection methods primarily focus on contextual information in the text and image information, neglecting other feature information, and simply concatenate features during fusion, which fails to fully utilize the multimodal information. To address these issues, we propose a multimodal rumor detection method based on topic features and multi-level fusion. First, contrastive learning is used to optimize the sentence embeddings of the pre-trained BERT model, enabling the model to learn high-quality embeddings that capture the semantics of sentences. Then, the optimized BERT model is employed to extract contextual semantic features from the text, while the LDA topic model is used to extract topic semantic features from the text. At the same time, the pre-trained ViT model is used to extract features from images, overcoming the limitation of small training samples. Next, a multi-level fusion of various features is performed to fully capture the potential relationships between different features. Finally, the obtained multimodal fused features are fed into the rumor detection module for classification. Experimental results show that our model outperforms mainstream multimodal baseline models on both Twitter and Weibo datasets, enhancing the effectiveness of multimodal rumor detection.