A unified approach for fake news detection: dynamic fusion of uni-modal and multi-modal features

Abstract The rise of multimedia content on social media has accelerated the spread of fake news, posing serious risks to social stability and eroding public trust. Therefore, detecting fake news has become a serious challenge. Current multi-modal fake news detection methods often fail to extract the basic frequency domain features of an image, and it is difficult to effectively extract and integrate the features of various modalities, and the weak inter-modal correlation and decision conflict have become the major performance bottlenecks. Designing effective multi-modal feature interaction strategies and adaptive modality weighting mechanism has become a core challenge to improve performance. To this end, we propose the Uni-modal and Multi-modal Fusion Network (UMFN). The network employs three branching encoders to capture textual, visual, and frequency domain features and a parallel multi-modal feature interaction strategy to explore the inter-modal dependencies to improve the fusion effect. In addition, using a pre-trained CLIP model, the network employs an adaptive modality weighting mechanism to balance the contributions of uni-modal and multi-modal features. This design mitigates weak inter-modal correlations and effectively alleviates decision conflicts arising from mismatched modalities. Experimental evaluations on two benchmark datasets show that the UMFN outperforms state-of-the-art methods, demonstrating its efficacy in feature extraction, interaction, and balancing. These findings emphasize the critical role of efficient multi-modal interaction and adaptive weighting in advancing fake news detection, laying the foundation for the development of more reliable detection methods.