CDFD: A Novel Cross-modal Dynamic Fusion and Self-distillation Approach in Fake News Detection

The quick dissemination of multimodal fake news in online communities has garnered significant attention due to its detrimental effects, prompting a focus on cross-modal detection of multimodal fake news. Several current methods aim to create multimodal representations by fusing single-modal features. However, existing methods mainly involved simple fusion of single-modal features, lacking dynamic fusion capabilities and global-to-local alignment. This may lead to poor quality in detecting multimodal fake news. To address these issues, this paper proposes a method called Cross-Modal Dynamic Fusion and Self-Distillation (CDFD) for fake news detection. The method aligns image and text embeddings through cross-modal contrastive learning. In addition, it introduces a multimodal Transformer to capture the joint representation of images and text. Dynamic fusion is achieved by employing an attention-guided weighting module to quantify the ambiguity between different modalities. Furthermore, soft labels are treated as additional training supervision to learn more expressive modal representations, and knowledge is transferred from the model to intermediate fusion modalities using self-distillation. Through evaluation and abatement studies on Weibo and Twitter datasets, it is experimentally demonstrated that the proposed method is able to learn features between multimodal features and outperforms the current state-of-the-art methods in fake news detection.