MMSFD: Multi-grained and Multi-modal Fusion for Short Video Fake News Detection

Short video platforms have become an essential part of people’s daily lives, while also facing the challenge of the rapid spread of fake news. Therefore, the detection of fake news in short videos has attracted increasing attention. Existing works mostly focus on the integration of textual and visual content, but fail to fully exploit multi-modal information at both fine-grained and coarse-grained levels. Moreover, multi-modal fake news detection is vulnerable to modality ambiguity due to potential conflicts or weak correlations between different modalities. In this paper, we propose a Multi-grained and Multi-modal fusion for Short video Fake news Detection (MMSFD). We encode textual and visual content using pre-trained models at both fine-grained and coarse-grained levels and fuse them through distinct branches. To tackle the ambiguity issue, we design a multi-modal fusion branch based on similarity weighting, as well as separate single-modal branches. Additionally, this paper presents a cross-modal attention mechanism for adaptively aggregating and reweighting features. The experimental results on a real-world short video dataset demonstrate that the proposed method outperforms state-of-the-art short video fake news detection methods.