A Statistical Learning Theory Analysis of Machine Learning and Human Judgment in COVID-19 Misinformation Detection

During the COVID-19 pandemic, the world faced an "infodemic",a massive wave of fake news and rumors that spread just as fast as the virus itself. To fight this, scientists have been trying to build computer programs that can automatically spot fake news. This study looks at three classic AI text-spotting tools (Logistic Regression, Naive Bayes, and LinearSVC) to figure out exactly how they work, how well they spot lies, and if they are actually better than everyday people at doing it.The paper is split into two parts. In the first part, we look deep under the mathematical hood of these AI models. We use math rules to see what we can,and cannot,actually promise about how well these models will handle brand-new text they've never seen before. We also fixed a common math mistake regarding how the Naive Bayes model assumes words are completely unrelated, showing that it can still occasionally work well even if its assumptions are wrong.In the second part, we put the models to the test on a big database of fact-checked COVID-19 articles and compared them to 50 regular people who took a 20-question fake-news quiz. One AI model (LinearSVC) did the best, getting a balanced accuracy score of 76.5%. Another model (Naive Bayes) looked great at first glance but turned out to be terrible at spotting the rare "true" articles because of its internal math biases. Ultimately, because the AI and the humans were graded on different sets of questions, we don't officially claim that the computers beat the humans. Instead, we layout exactly what a fair, head-to-head match would look like for future research.