Sentiment Analysis Through Prompt Engineering with GPT-4 on the SmSA IndoNLU Dataset

This research investigates the application of prompt engineering techniques for sentiment analysis using GPT-4 and BERT models on the IndoNLU SmSA dataset. The BERT model is fine-tuned on the dataset, while GPT-4 is employed using various prompt engineering techniques, including zero-shot, one-shot, few-shot, chain-of-thought, and a fusion technique that integrates structured reasoning with factual identification. Both models are evaluated using accuracy, precision, recall, and F1-score. The results reveal that the fusion technique for GPT -4 outperforms other methods, achieving an accuracy of 96.8% and an F1-score of 0.968, significantly surpassing the performance of the fine-tuned BERT model (accuracy: 90.6%). This superior performance is attributed to the fusion technique's ability to guide the model through step-by-step reasoning, enhancing its capacity to handle complex language features like sarcasm and irony. Practical implications of these findings include the potential for more accurate and reliable sentiment analysis in low-resource languages such as Indonesian, with applications in business intelligence, social media monitoring, and customer feedback analysis. Additionally, the study highlights the importance of prompt design in improving model performance and suggests that combining multiple prompt techniques can significantly enhance sentiment classification accuracy. The research also underscores the need for robust security measures, successfully mitigating prompt injection attacks by reducing vulnerability rates from 25.3% to 3.8%. The implications for enhancing AI-driven sentiment analysis in low-resource languages, with opportunities for future research to explore local models and hybrid prompt strategies.