Prediction of fake news on X: An analysis of LLMs and machine learning methods on TruthSeeker

Communication has developed with technology and become a global network. Social media platforms like X make it very easy to share information. This causes fake and misleading news to spread rapidly in society. Because in these new ways of obtaining information, some difficulties arise in determining the reliability and accuracy of information. In order to overcome these difficulties, machine learning and natural language processing techniques play an important role today. In this study, a highly reliable and effective approach has been presented using the TruthSeeker dataset, large language models (LLM) and machine learning (ML) techniques to detect fake news on the X platform. Comparative and different variation tests of 12 different ML models and 6 LLMs have been performed for the proposed model and it is aimed to detect whether the news is fake or not binary. As a result of the experiments, 92.8% accuracy value was obtained with BERTweet and the original Soft Voting Ensemble structure, while 92.9% F1-score value was achieved. The 10-Fold CV-AUC value was 0.976 on average and its standard deviation was 0.001. This proves that the data dependency of the proposed model is quite low. In addition, the obtained results showed that the model can detect fake news with higher success than real news. The obtained results show that BERTweet and Voting Soft Ensemble models can be effective and reliable tools in the task of fake news detection. In addition, with the proposed model, it will be possible to prevent the spread of misleading news and protect users, and the social impact of these news will be reduced.