Leveraging LLMs for LLM-Generated Fake News Detection: Insights from COVID-19 Misinformation

The rapid spread of misinformation on social media, particularly during crises like the COVID-19 pandemic, underscores the urgent need for effective detection systems. Large language models (LLMs) have emerged as powerful tools for combating misinformation due to their extensive world knowledge and strong reasoning abilities. However, LLMs also present challenges to misinformation detection systems, as they can be misused to generate highly persuasive disinformation. This study investigates the dual role of LLMs as both generators and evaluators of fake news, focusing on four models: GPT-3.5, GPT-4, Gemini, and Claude. We evaluate their ability to detect fake news within mixed datasets of real and fake news, generated by LLMs. Our findings highlight the strengths and limitations of modern LLMs in identifying misinformation in short-form contexts, offering valuable insights into mitigating the spread of fake news.