Urdu-Punjabi Code Switched Sentiment Analysis Empowered by a Deep Learning Framework Integrating XLM-R, and GPT

Sentiment analysis is a procedure that uses computational methods, textual analysis, and natural language processing to derive significant insights from textual sources. Sentiment analysis detects and quantifies the attitudes, opinions, and emotional states that individuals convey through textual information. The current sentiment analysis research has predominantly focused on the English language, resulting in a significant lack of attention towards low-resource languages. Performing sentiment analysis on low-resource languages is challenging due to the unavailability of extensive datasets and associated resources. To overcome the challenge of unavailability of datasets we proposed Large Urdu-Punjabi code switched Corpus for Sentiment Analysis (LUPCSA-25) comprises over 10,00,000 user reviews in Urdu and Punjabi (Shahmukhi). Urdu and Punjabi domain specialists enrolled in PhD provided additional annotations to the dataset. In this study we explore the efficacy of head pruning techniques for improving the performance and computational efficiency of transformer-based models, specifically XLM-R and GPT-2, for sentiment analysis of Urdu-Punjabi code switched text. We preprocessed the input text, generate the BERT embeddings, and then feed them into the proposed classifier for sentiment classification. The performance of the proposed classifier is assessed by comparing it with baseline classifiers. The results demonstrate that the proposed classifiers with head pruning technique surpass current state-of-the art models with a precision rate of 96.4%.