A Comparative Study of BiLSTM and RoBERTa Model for Sarcasm Detection

Detecting sarcasm in text remains a complex challenge in Natural Language Processing (NLP) due to its subtlety and reliance on context. This research compares the effectiveness of two deep learning models—Bidirectional Long Short-Term Memory (BiLSTM) and the Robustly Optimized BERT Pretraining Approach (RoBERTa)—for identifying sarcasm in news headlines. Leveraging the Sarcasm Headlines Dataset v2, which contains 28,619 labeled samples, both models were assessed using metrics like accuracy, F1-score, and loss over several training epochs. Findings show that RoBERTa significantly outperforms BiLSTM, achieving a validation accuracy of 93.76% and an F1-score of 0.9326 at epoch 10, compared to BiLSTM’s 84.70% accuracy and 0.8403 F1-score. RoBERTa’s advantage stems from its transformer-based architecture and self-attention mechanisms, enabling better understanding of contextual nuances, whereas BiLSTM's sequential nature limits its grasp of long-range dependencies. The study underscores the critical role of model architecture in sarcasm detection and discusses the computational considerations involved. Future directions include exploring hybrid models, incorporating multimodal data, and applying domain-specific fine-tuning to improve detection performance further.