Interpretable BERT and RoBERTa based Fake News Detection using LIME

Fake news has become a widespread issue today, with the vast use of social networks further accelerating its spread. The research community is actively working on tackling this challenge with the help of artificial intelligence, machine learning, and deep learning. Transformer-based model is a type of deep learning architecture that relies on the self-attention mechanism and has attracted significant research attention due to their ability to understand and capture contextual information effectively. However, transformer-based models encode information in a highly distributed manner across several layers, which results in a lack of model interpretability. It remains unclear how the transformer-based models determine whether news is real or fake, a challenge commonly referred to as the black-box problem. In this study, we fine-tune transformer-based models, BERT and RoBERTa, on the benchmark dataset FakeNewsNet to identify the most influential features that address the interpretability challenge. We use LIME (local interpretable model-agnostic explanations) to add interpretability to the fine-tuned models. The fine-tuned models achieve accuracies of 98.16% for BERT and 99.20% for RoBERTa, surpassing state-of-the-art performance by a margin of 3%. Furthermore, the use of LIME provides justifications for the predictions of each model, enhancing the trust of users in AI-driven fake news detection systems.