A fuzzy-based multimodal approach for interpretable fake news detection

Fake news has become a significant threat to society, exacerbated by the rapid growth of multimedia content on social media platforms. While several existing approaches focus on integrating textual and visual data for fake news detection, they often rely on traditional fusion techniques such as concatenation or addition, which suffer from two primary issues, i.e. limited cross-modal feature fusion, and inadequate detection of inconsistencies between text and images. To address these challenges, this paper proposes a novel fuzzy-based multimodal fake news detection framework that integrates text, images, and their corresponding semantic relationships for more accurate and interpretable results. The model combines a Long Short-Term Memory for text analysis, an Adaptive Neuro-Fuzzy Inference System for image classification, a similarity analysis to check the coherence between text and image, and a Fuzzy Inference System to reason about the final outcome. This multimodal fusion allows the system to detect subtle inconsistencies that may signal the presence of fake news. Furthermore, this framework produces interpretable results by describing the particular contributions of each modality (text, picture, and semantic consistency) to the final categorization. Extensive experiments on three publicly available datasets (Twitter, Buzzfeed, and PolitiFact) demonstrate that the proposed method significantly improves fake news detection performance, achieving superior accuracy, precision, recall, and F1 scores compared to state-of-the-art methods. The study is completed by a formal analysis of the Fuzzy Inference System, to support a brief discussion on its stability. • Deployment of fuzzy and non-fuzzy techniques into a novel framework. • Text, image and image caption consistency processed for fake news detection. • Interpretable scheme unlike most state-of-the-art methods. • Properties of the fusing system presented to support the stability check. • Several experiments performed on three publicly available datasets.