A Comparative Analysis of ChatGPT and Traditional Machine Learning Algorithms on Real-World Data

The rapid growth of computer-based technologies has transformed many sectors, with artificial intelligence playing a key role in automating tasks previously performed by humans. In this context, natural language processing models such as chatbots, including Chat Generative Pre-Trained Transformer (ChatGPT), are increasingly being used as analytical tools alongside traditional machine learning algorithms. However, despite these advancements, concerns remain regarding the accuracy, processing time, and overall reliability of ChatGPT compared to traditional coding-based machine learning algorithms. This study provides a comparative evaluation of ChatGPT’s ability to generate intelligent responses. It focuses on three key aspects: accuracy across various datasets at different time intervals using the same account, performance relative to traditional machine learning algorithms in terms of accuracy, and the variability of ChatGPT’s results across diverse data sources. To address these concerns, 15 algorithms were tested against ChatGPT. Tests were done at four different time intervals using healthcare and education datasets. ChatGPT showed competitive accuracy but had more variability and slower processing. As a result, this study highlights notable performance limitations for ChatGPT. For instance, in the heart disease dataset, the Random Forest model achieved an accuracy of 0.672 in 0.012 seconds, whereas the average performance of ChatGPT was 0.608 with a processing time of 0.274 seconds. In comparison, the traditional Gradient Boosting Machine model attained an accuracy of 0.623 in 0.124 seconds, while ChatGPT recorded an accuracy of 0.589 in 1.019 seconds. Finally, this study draws specific conclusions based on the results and offers recommendations for future research.