MSFF: A Novel Multi-Modal Semantic Feature Fusion for Social Media Fake News Detection

Social media has become the primary channel for spreading fake news content, and the form of fake news is gradually evolving from a single form to multi-modal forms including text, voice, images and videos etc. This paper proposes a novel multi-modal semantic feature fusion (MSFF) for social media fake news detection based on text semantics and visual entity representation methods. Through this method the text and image unimodal features are first classified and extracted, Next the BERT language model is utilized for text features and VGG19 convolutional neural network utilized for image features to vectorize representation. Then the multi-modal features are merged and they are mapped into semantic consistency feature labels through the attention network. Finally weighted joint detection results for fake news are reached. Experimental validation conducted on PolitiFact and GossipCop in the fake news detection dataset FakeNewsNet demonstrates that the MSFF model increases the accuracy rate and recall rate by 2.6% and 5.6% respectively compared to the current state-of-the-art methods, which indicates that the method can fully integrate multi-modal feature information and effectively enhance the performance of detecting fake content in social media news.