A Comparative Study of Alternative Automatic Labeling Using AI Assistant

The development of AI assistants has become increasingly sophisticated, as evidenced by their growing adoption in assisting humans with various tasks. In particular, AI assistants have demonstrated potential in the field of sentiment analysis, where they can automate the labeling of text data. Traditionally, this labeling process has been performed manually by humans or using tools like the VADER Lexicon. This study is imperative to evaluate the performance of AI Assistants in sentiment labeling, as compared to traditional human-based labeling and the application of the VADER sentiment analysis algorithm. The methodology involves comparing the labeling results of Gemini and You AI with those of human labeling and VADER. Performance is evaluated using the Naive Bayes and K- Nearest Neighbour algorithms, and K-Fold Cross Validation is employed for evaluation. The results indicate that the performance of both AI assistants can closely approximate the performance of human labeling. Gemini's best accuracy is achieved with the k-NN algorithm at 53.71%, while You AI's best accuracy is achieved with the Naive Bayes algorithm at 48.30%. These results are close to the accuracy of human labeling (61.12%) using the k-NN algorithm and VADER (54.29%) using the Naive Bayes algorithm. This suggests that AI assistants can serve as an alternative for text data labeling, as the differences in performance are not statistically significant.