A multi-level fusion-based framework for multimodal fake news classification using semantic feature extraction

Abstract In the era of rapid digital news development, fake news poses a severe threat to society’s determination and authenticity, especially with the advent of online media platforms that facilitate the creation and dissemination of fabricated information. Although various techniques have been developed to discriminate between authentic and fake news, a practical fake news classification framework is still needed to automatically deliver high classification performance and impede the spread of misinformation. To fill this gap, this study proposes a multi-level fusion-based CNN with dual-conv layers-RNN (CDLR) framework that fuses Convolutional Neural Networks (CNN) with dual Conv layers, Recurrent Neural Networks (RNN), and classification module for multi-model fake news classification. The proposed framework fuses CNN (with dual-Conv layers) and RNN to enhance classification abilities and extract high-quality semantic textual and visual features to identify misinformation effectively. After pre-processing, the extracted weight matrix was fed to CNN (with dual Conv layers) to learn and extract deep visual features and an RNN for high-quality feature extraction from textual data or news articles for classification. Likewise, we designed a fusion mechanism to cross-validate the execution of our framework by considering different variants such as mean fusion, weighted-mean fusion, maximum fusion, and sum fusion. Finally, a classification module with a polynomial kernel was employed to categorize the extracted data as fake or real, for final classification. A comprehensive experiment analysis was carried out to evaluate the proposed framework's effectiveness by combining early and late fusion mechanisms with baseline methods on five extensive, fair, and diverse datasets. The accuracy of our framework was found to be 0.9725 on ISOT, 0.9107 on Fake vs. Real News, 0.9816 on WELFake, 0.5403 on FA-KES, and 0.9163 on Twitter datasets, indicating its robustness in classifying fake news compared to benchmark methods. Lastly, the study proposes some recommendations to mitigate the adverse effects of fake news based on the predictions made using our fusion-based framework.