Combating Fake News on Social Media: An Early Detection Approach Using Multimodal Adversarial Transfer Learning

The proliferation and rapid spread of fake news on social media pose a significant threat to society, underscoring the urgent need for effective early detection methods. This paper introduces multimodal adversarial transfer learning (MATRAL), a novel approach designed for early fake news detection. MATRAL integrates multimodal learning with adversarial transfer learning. Through effective multimodal learning, MATRAL can form a comprehensive representation of news items on social media, including text, images, and publisher information. The adversarial transfer learning component enables MATRAL to dynamically adapt its knowledge to new domains, ensuring the approach’s ongoing relevance against the evolving fake news generation tactics. Using the MediaEval 15–16 data sets to simulate the early fake news detection scenario, we conduct extensive experiments to evaluate MATRAL’s performance against state-of-the-art methods in multimodal fake news detection, machine learning methods, and industrial practices. The experimental results conclusively demonstrate MATRAL’s superiority across various widely adopted metrics, showcasing its proficiency in early stage fake news detection. To further elucidate the contributions of various model components, a series of ablation studies are conducted. Furthermore, MATRAL’s interpretability and robustness are substantiated through additional experimental analyses. Our work introduces a novel and robust solution to the pressing challenge of multimodal fake news detection on social media, offering a significant contribution to the research and practice of responsible artificial intelligence. History: This paper has been accepted by Kaushik Dutta for the Special Issue on the Responsible AI and Data Science for Social Good. Funding: This research was partially supported by the National Natural Science Foundation of China [Grants 72495123, 72101007, and 72201061]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.0514 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2023.0514 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .