MDTF: Multimodal Rumor Detection with Dual Augmentation at Textual and Feature Levels

While social media facilitates communication, it also contributes to the widespread dissemination of rumors, posing significant societal harm. Although numerous detection methods have been proposed to address online rumors, most lack efficient feature fusion for multimodal rumor detection, leading to suboptimal detection performance. Therefore, we introduce the Multimodal Rumor Detection with Dual Augmentation at Textual and Feature Levels model (MDTF), which utilizes feature extraction modules to separately extract textual and visual features. The model employs a cross-attention mechanism to enhance visual features and incorporates a gradual feature fusion approach. Considering the potential signal interference and feature coverage issues from feature fusion, we implement residual connections to mitigate these effects. Additionally, due to the limited data volume in rumor detection datasets, we employ a text data augmentation module to generate a substantial amount of text based on images, thereby enriching the textual dataset. To evaluate the performance of the MDTF model, experiments were conducted on the Twitter dataset, demonstrating that the MDTF model surpasses the most advanced models in terms of accuracy and F1 score.