Performance Comparison of Sentiment Classification Algorithms on SIGNAL Reviews Using SMOTE

Public service apps like SIGNAL are widely used to provide public access to information and vehicle tax payments. However, diverse user reviews highlight the need to evaluate public perception through sentiment analysis. Selecting an appropriate classification algorithm is crucial to ensure accurate results, particularly when dealing with imbalanced review data. Therefore, This study examines the comparative performance of four algorithms Naïve Bayes, Random Forest, Decision Tree, and SVM in analyzing the sentiment of 36,000 user feedback obtained from Google Play Store. The dataset underwent preprocessing, feature extraction using TF-IDF, and class balancing using SMOTE. Model evaluation was conducted using accuracy, precision, recall, and F1-score. The findings indicated that Random Forest performed the best overall performance (accuracy 91.04%, F1-score 94.80%), followed by Naïve Bayes (accuracy 89.89%, F1-score 93.38%), SVM (accuracy 89.22%, F1-score 93.02%), and Decision Tree (accuracy 88.40%, F1-score 92.31%). These findings indicate that Random Forest is highly effective for balanced datasets, while SVM and Naïve Bayes offer competitive precision for applications prioritizing accuracy in positive class detection. The output of this study can be applied practically by developers and related institutions in optimizing public service applications and by applying Random Forest algorithm to gain actionable insights for optimizing features and aligning services more closely with user needs.