Sentiment Analysis of User Reviews of the AdaKami Online Loan App from the App Store Using SVM and Naive Bayes

This study aims to classify sentiments on user reviews of the AdaKami online loan application, which are obtained through web scraping techniques from the Apple App Store platform. A total of 2000 reviews were collected, then selected and 1000 reviews were selected to be manually labeled by two linguistic experts, to ensure the validity of the classification. Sentiments are divided into three categories, namely negative, neutral, and positive. The classification model was built using two machine learning algorithms, namely Support Vector Machine (SVM) and Naïve Bayes (NB). The evaluation was carried out by measuring accuracy, precision, recall, F1-score, as well as through confusion matrix and cross-validation. The results showed that SVM performed better, with an accuracy of 97.5%, an F1-score of 0.97, and an average cross-validation accuracy of 84.69%. In contrast, Naïve Bayes recorded an accuracy of 81.4% and an F1-score of 0.77. The results of the paired t-test showed that the difference in performance between the two models was statistically significant (p < 0.05). The SVM model was then applied to predict 971 unlabeled reviews, and the results showed a dominance of negative sentiment. Wordcloud visualizations reinforced this finding, with words such as “bilih”, “bunganya”, and “teror” as the most frequently occurring words. These findings prove that SVM is more effective in classifying online loan review sentiments, as well as providing important insights for developers in understanding user perceptions and experiences.