GINMCL: graph isomorphism network-driven modality enhancement and cross-modal consistency learning for multi-modal fake news detection

Multi-modal fake news detection is a technique designed to identify and classify fake news by integrating information from multiple modalities. However, existing multi-modal fake news detection models have significant limitations in capturing structural information when processing social context. The core issue stems from the reliance on simple linear aggregation or static attention mechanisms in existing graph detection methods, which are inadequate for effectively capturing complex long-distance propagation relationships and multi-layered social network structures. Furthermore, existing multi-modal detection approaches are limited by the feature representations within the respective semantic spaces of each modality. The semantic gaps between modalities lead to misalignment during information fusion, making it difficult to fully achieve modality complementarity. To address these issues, we propose GINMCL, a graph isomorphism network-driven modality enhancement and cross-modal consistency learning method for multi-modal fake news detection. This method builds on the extraction of text and image features by incorporating graph isomorphism networks (GIN) based on the Weisfeiler-Lehman (WL) injective aggregation mechanism to effectively capture both local dependencies and global relationships within social graphs. Modality consistency learning aligns text, image, and social graph information into a shared latent semantic space, enhancing modality correlations. Additionally, to overcome the limitations of traditional methods in modality fusion strategies, we leverage a hard negative contrastive learning mechanism, which softens the penalty on negative samples and optimizes contrastive loss, further improving the accuracy and robustness of the model. We conducted systematic evaluations of GINMCL on the Pheme and Weibo datasets, and experimental results demonstrate that GINMCL outperforms existing methods across all metrics, achieving state-of-the-art performance.