Unraveling Misinformation: Insights from Large-scale Data Analysis

The proliferation of fake news on social media prevents users from obtaining accurate information, affects people's judgment, endangers public health, and undermines social stability, often with serious consequences. We collected 226,681 instances of misinformation from major media outlets such as Yahoo, Fox News, and Forbes, spanning from August 21, 2018, to August 21, 2023. Our analysis aims to uncover common patterns, popular themes, and potential factors that lead to the widespread of misinformation. We utilized a large language model (LLM) to address potential biases in the data and assess four key bias dimensions: gender, political, racial, and religious. This in-depth analysis revealed complex bias patterns across various media platforms. In our comparative evaluation of machine learning models–including logistic regression, random forest, RNN, CNN, and SBERT-we observed promising results in misinformation classification. News headlines contain highly informative signals, yielding strong predictive performance. While incorporating full-text analysis offered marginal improvements in accuracy, the headline alone emerged as a crucial predictor, underscoring the dense informational value of concise text. The complete dataset is available at: https://doi.org/10.5281/zenodo.11314826.