Multimodal Rumor Detection Based on Co-Attention Mechanism

The prevalent spread of rumors may mislead readers and bring negative impacts. Detecting rumors on social media platforms has become a top priority. A variety of algorithms have been proposed to help detect rumors, ranging from manual feature extraction to deep learning methods. However, these methods have limitations and fail to fully utilize the multimodal information in posts or the intrinsic connections between multimodal features. To address these problems, this paper proposes a multimodal rumor detection model based on co-attention mechanism (MMCM). Specifically, we use Resnet and Bert to extract image pixel features and text features, design a CNN network model to extract frequency domain features of images, and finally fuse these features through a feature fusion module based on a collaborative attention mechanism for rumor detection. Experiments on two public benchmark datasets (Twitter and Weibo) verify that the performance of the proposed rumor detection method is better than that of existing methods.