Analysis of the Effect of Attention Mechanism on the Accuracy of Deep Learning Models for Fake News Detection

The main objective of the paper is to verify whether the integration of attention mechanisms could improve the effectiveness of online fake news detection models. The models were training using selected deep learning methods, which were suitable for text processing, such as CNN (Convolutional Neural Network), LSTM (Lon-short Term Memory), BiLSTM (Bidirectional LSTM), GRU (Gated Recurrent Unit), and transformer. The novelty of the paper lies in the addition of attention mechanisms to each of those models, and comparison of their performance across both datasets, LIAR and WELFake. Afterwards, an analysis of resulting changes in terms of the detection performance was carried out. The paper also describes the issue of toxicity in the online space and how it affects society, the toxicity sources, and methods to tackle it. Furthermore, the article provides a description of individual deep learning methods and the principles of attention mechanism. Finally, it was shown that the attention mechanism can increase the accuracy of basic models for fake news detection; however, the differences are insignificant in the case of the LIAR dataset. The reason for this can be found in the dataset itself. In contrast, the addition of attention mechanism to models on the WELFake dataset showed a significant improvement of results, where the average accuracy was 0.967 and average F1-rate was 0.968. These results were better than the results of experiments with the simple transformer. Comparison of the results showed that it makes sense to enrich the basic neural network models with the attention mechanisms, especially with the multi-head attention mechanism. The key finding is that attention mechanisms can enhance fake news detection performance when applied to high-quality, well-balanced datasets.