Bidirectional Encoder Representations from Transformers Fine-Tuning for Sentiment Classification of Cek Bansos Reviews

Social assistance programs are essential government initiatives aimed at supporting underprivileged communities. One such program is facilitated through the Cek Bansos application, which enables users to check their eligibility for social aid. However, user experiences with the application vary, leading to a range of sentiments in their reviews. Understanding these sentiments is crucial for improving the application’s functionality and user satisfaction. This study focuses on sentiment analysis of user reviews of the Cek Bansos application by leveraging a fine-tuned Indonesian-language Bidirectional Encoder Representations from Transformers (BERT) model. The aim of this research is to evaluate the effectiveness of the BERT model in classifying sentiments in user reviews and to provide insights that could improve the Cek Bansos application. This research method is the BERT model was fine-tuned using hyperparameters such as a learning rate of 3e-6, batch size of 16, and 9 epochs. The dataset consisted of 8,000 reviews, divided into training (70%), validation (20.1%), and test (9.9%) sets. Review scores were manually categorized, where ratings of 1 to 2 were classified as negative sentiment, 3 as neutral, and 4 to 5 as positive. The results of this research are as follows: the fine-tuned model achieved an accuracy of 77%, with additional evaluation metrics such as precision, recall, and F1 score, demonstrating the model's effectiveness in identifying positive, negative, and neutral sentiments separately. The conclusions of this study are that the BERT model provides a reliable method for sentiment classification of user reviews, which could support developers and policymakers in refining the Cek Bansos application to enhance user experience. Additionally, a web-based application developed using Streamlit allows government officials to visualize sentiment trends in real time, improving their understanding of user feedback. Future research could explore alternative machine learning models and additional linguistic features to further improve sentiment classification accuracy and the overall user experience.