Who Detects Better? A Comparative Study on Misinformation Detection by Humans and Large Language Models

Large language models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, their ability to detect and react to misinformation remains an open question, particularly in comparison to human cognitive mechanisms. This study investigates how LLMs and humans react to misinformation by analyzing their performance across five categories of errors: intellectual, common sense, reasoning, misleading, and logical errors. We construct the ErrorQuestionDataset, comprising 346 misinformation-related questions, and conduct an empirical study involving five state-of-the-art LLMs (ChatGPT-4o, Gemini-1.5flash, DeepSeek-v3, Hunyuan-Large, GLM-v4Flash) and 251 human participants. Our findings reveal distinct response patterns: while LLMs rely on statistical correlations and pattern recognition, humans leverage contextual reasoning and domain-specific knowledge. The results indicate that LLMs generally achieve higher accuracy than humans in error detection tasks, but their performances lack depth in reasoning-based assessments. Additionally, we identify five primary performance types—affirmation, negation, hesitation, questioning, and off-topic reactions—providing insights into the cognitive differences between LLMs and human cognition. Our study contributes to the broader understanding of misinformation detection and offers implications for enhancing the robustness and reliability of LLMs in real-world applications. Our code and dataset are available at https://github.com/anonymous-submission8888.