Exploring the Impact of Different Textual Language Features on Large Language Models' Detection of Fake News

Abstract: With the proliferation of social media and online platforms, fake news has become increasingly rampant. This study explores the impact of different textual language features on large language models (such as ChatGPT) in detecting fake news. By extracting extreme emotional vocabulary and exaggerated syntactic words commonly found in fake news and calculating their TF-IDF values, this study analyzes their influence on large language models' ability to assess the veracity of news. The study found that the frequency of extreme emotional words is higher than that of exaggerated syntactic words and has a more significant impact on fake news detection by large language models. Furthermore, this study suggests that by carefully selecting and adjusting language features, the accuracy and stability of fake news detection can be improved, providing new insights for optimizing automated detection systems. These findings provide important references for improving the technology of automatic fake news detection, contributing to the construction of a safer and more reliable online environment.