Sentiment Analysis of Public Complaints: A Machine Learning Comparison of SVM, Naive Bayes, Random Forest, and XGBoost

Public complaints serve as a crucial indicator for understanding various social issues faced by Indonesian society. In this context, the abundant reports submitted through the Suara Surabaya (@e100ss) platform, a dedicated account focusing on transportation and traffic, require immediate attention to ensure a quick and effective response, as they pertain to road safety. Therefore, a method is needed to classify report texts into complaints and non-complaint categories. This study aims to filter complaint data from the Suara Surabaya (@e100ss) account while comparing text mining models by applying Natural Language Processing (NLP). Using this technique, the time required to identify complaints can be significantly reduced, eliminating the need to manually review each report. The research data were obtained through a Twitter data crawling process from the Suara Surabaya (@e100ss) account, covering the period from February to December 2023, resulting in approximately 2,500 classified data entries. The labeling of complaint and non-complaint data is done manually based on ground truth. A comparison of models utilizing NLP was conducted to determine which model achieves the highest accuracy in the context of complaint classification. The findings reveal that the XGBoost model achieved the highest accuracy at 0.86, followed by the Support Vector Machine (SVM) with 0.84, Random Forest with 0.78, and Naive Bayes with 0.77. This research is expected to contribute to accelerating the handling of public complaints on the Suara Surabaya platform and serve as a reference for other researchers studying sentiment analysis within the same context.