Evaluating and fine-tuning CAMeL-BERT for analyzing customer sentiment in Saudi dialect

Social media platforms, particularly Twitter (X), have become significant channels for users to express opinions and provide feedback on various services. In Saudi Arabia, where Twitter (X) is widely used, the platform serves as a valuable source of real-time sentiment data regarding private and public sector services. Understanding user sentiments can provide organisations and companies with insights that help improve services and customer satisfaction. One effective way to measure sentiment is through the use of pre-trained models. While sentiment analysis models for English are well established, their effectiveness cannot be assumed for other languages. The linguistic structure, idiomatic expressions, and cultural nuances of Arabic require a model trained primarily on Arabic data. Moreover, pre-trained Arabic models are often built using architectures designed for English models. These Arabic models are generally trained on smaller datasets compared to the vast amounts of data used for training English models. This gap in training data leads to reduced performance and accuracy for Arabic models, as they lack the extensive linguistic and contextual understanding that English models gain from larger datasets. The CAMeL-BERT pre-trained model is specifically designed for Arabic natural language processing (NLP) and is currently one of the most advanced and accurate models for handling Arabic text. In this study, we aim to evaluate the performance of the CAMeL-BERT model in comparison with an English-based model and to enhance its ability to understand the Saudi dialect through fine-tuning. We apply this approach to data collected from Twitter (X), focusing on customers’ tweets regarding the services of the Saudi Telecom Company (STC), the largest telecommunications company in Saudi Arabia. The results highlight the importance of model adaptation for specific languages and contexts and underline the potential of CAMeL-BERT in sentiment analysis for Arabic-language content. The findings offer practical implications for enhancing customer service and engagement through more accurate sentiment analysis of social media content in the service providers sector.