Performance Comparison of Naïve Bayes and SVM Algorithms in Sentiment Analysis on JKN Application Data

In 2022, 67.88% of Indonesia's population owned mobile devices. BPJS Kesehatan responded to this trend by launching the Mobile JKN application to provide modern, accessible healthcare services. To drive continuous innovation, BPJS Kesehatan needs insights into user feedback regarding the Mobile JKN application. Given the large volume of reviews, sentiment analysis is employed to classify reviews into positive or negative categories. This study compares the performance of Naïve Bayes and SVM (Support Vector Machine) algorithms in sentiment classification using a dataset from the Mobile JKN application. The dataset consists of 200 reviews labeled by two different raters, yielding 110 positive and 90 negative reviews for the first set and 114 positive and 86 negative reviews for the second set. Testing was conducted using three data split scenarios for training and testing: 70:30, 80:20, and 90:10. Model performance was evaluated using a confusion matrix, with metrics including accuracy, precision, recall, and F1-score. The results show that the Naïve Bayes algorithm achieved its best performance with a 90:10 data split, yielding an accuracy of 85%, precision of 77%, recall of 100%, and F1-score of 87%. Conversely, the SVM algorithm performed best with an 80:20 data split, achieving 93% accuracy, 100% precision, 84% recall, and an F1-score of 91% for the first rater's dataset. For the second rater's dataset, SVM reached optimal performance with a 90:10 data split, yielding 90% accuracy, 100% precision, 80% recall, and an F1-score of 89%. Overall, the comparison highlights that SVM outperforms Naïve Bayes in terms of accuracy and precision, making it more effective for predicting positive sentiment in Mobile JKN application reviews.