Sentiment Classification of IT Service Feedback via TF-IDF

Handling user complaints and feedback is a key strategy of Pusintek, the Ministry of Finance of the Republic of Indonesia, to enhance user satisfaction. The challenge faced is the difficulty in accurately analyzing feedback due to differences in comments and categories chosen by users, which requires manual category correction. This study aims to automate feedback comment categorization using classification algorithms. Specifically, Naïve Bayes, Support Vector Machine (SVM), and K-Nearest Neighbors (K-NN) algorithms were applied to 11,108 user feedback records. The CRISP-DM framework was used, with dataset preparation involving sentiment analysis techniques (cleansing, case folding, normalization, filtering, and tokenization) and Term Frequency-Inverse Document Frequency (TF-IDF) weighting. Accuracy values for each algorithm were evaluated. Results show that the SVM algorithm performed the best, achieving an accuracy of 94.10% and consistently delivering the highest precision, recall, and f1-score across all sentiment categories. This research contributes to the development of an automatic feedback classification system that improves categorization accuracy, minimizes manual intervention, and optimizes user feedback analysis. It is expected to enrich the understanding of text classification and natural language processing techniques and open up opportunities for further research.