Leveraging Synthetic Data and LLMs for Sarcasm Detection

Sarcasm's nature of inverting the implied sentiment makes it difficult to perceive. While sarcasm detection has seen significant development in recent years, most research has focused on the English language, limiting its applicability to sentiment-related tasks in low-resource languages like Filipino. In this work, we address this gap by leveraging synthetic data and large language models (LLMs) to create a textual Filipino sarcasm detection dataset and assess LLM performance on the task. We utilize LLMs due to their capability to generate data quickly with controlled distribution at a much lower cost and the opportunity to test their understanding of the subtle forms of sarcasm, as well as the Filipino language's linguistic and cultural nuances. Our dataset of 1008 synthetic and 100 real-world tweets enabled Gemini 2.0 Flash to achieve 96% accuracy with few-shot prompting on synthetic data. However, performance dropped to 74% on real-world data with zero-shot prompting. This discrepancy highlights the limitation of synthetic data in capturing the authenticity of real-world language and points to the need for fine-tuning LLMs to better adapt to the nuances of Filipino sarcasm detection.