Toward Explainable Multimodal Fake News Detection: A Fusion of Prompt Learning and Automated Machine Learning

Multimedia fake news detection faces challenges due to the complexity of misinformation, requiring large datasets, fine-tuning, and explainability. This paper proposes an explainable multimodal framework that integrates textual and visual modalities. The image pipeline leverages feature extraction, Fuzzy C-Means clustering, and AutoGluon for robust classification. The text pipeline enhances Large Language Models with prompt learning, integrating domain knowledge for accurate fake news detection. Outputs from both pipelines are fused using a Functional Network, ensuring explainability and addressing interpretability issues. The stability of this fusion scheme is formally discussed. Extensive experiments on three benchmark datasets demonstrate that our proposed model outperforms existing methods on multimodal fake news detection. The average improvement in accuracy through the different datasets is up to 10%, in addition to the fact that the proposed multimodal approach can address the questions of explainability and interpretability, unlike the existing methods.