Sentiment analysis of Nepali social media text with a hybrid deep learning model

Abstract Sentiment analysis (SA) has been a well-researched area in text mining, and significant progress has been made recently. Both traditional machine learning (ML) and recent deep learning (DL) methods have shown significant accomplishments in the SA for resource-rich languages. Various DL models, such as convolutional neural networks (CNNs), long short-term memory (LSTM), Bidirectional LSTM (Bi-LSTM), and pre-trained language models (PLMs), have been successfully used for SA in resource-rich settings. However, these models often struggle with low-resource languages like Nepali, primarily due to the limited availability of pre-trained models for handling complex language structures such as rich morphology, short texts, and unbalanced datasets. We propose a hybrid DL model that leverages contextual features from a PLM and spatial features from a convolutional module. For this, pre-trained multilingual embeddings from XLM-RoBERTa are utilised to comprehend the language context, which is then gated with a 1D convolution for local spatial pattern extraction. Finally, the softmax activation is applied with a dense layer to detect positive, negative, and neutral sentiments. We evaluate our model on three diverse Nepali SA datasets (D1, D2, and D3) and find the highest accuracy of 74.77% (D1), 79.52% (D2) and 55.82% (D3) compared to the state-of-the-art (SOTA) SA models for Nepali text.