A Hybrid Approach To Solving The Sarcasm Detection Problem: BERT vs GPT-BERT

With the boom of Natural Language Processing (NLP) machines like ChatGPT in recent years, the demand for accurate sarcasm detection has surged.However, to the best of our knowledge, there is not an existing approach that addresses the three main challenges of sarcasm detection simultaneously: understanding context, implicit references, and common sense.Hence, we propose a novel Deep Learning (DL) based hybrid approach to tackle these challenges.We utilize Generative Pre-Trained Transformer-3.5 Turbo (GPT-3.5 Turbo) for appending additional information in each dataset example if any historical, cultural, or political reference or renowned figure is identified.This modified dataset is used to train Bidirectional Encoder Representations from Transformers (BERT).We will be using a large, well-labelled News-headlines dataset for training and testing.A current study claims to have obtained 91.47% accuracy on the same dataset using BERT.This will be used as the baseline to compare with our hybrid approach (GPT-BERT) and a BERT-based approach with modified hyperparameters.The GPT-BERT approach resulted in greater precision and lower validation loss of 0.94% each, compared to the BERT-based approach.However, it underperformed in terms of accuracy (by 0.6%), recall (by 2.5%), and F1 score (by 0.78%).Nevertheless, both approaches outperformed the baseline.