Comparison of the Performance of Gradient Boosting and Extreme Gradient Boosting Methods in Classifying Timms Science Achievement

This study aims to compare the classification performance of machine learning methods Gradient Boosting (GB) and Extreme Gradient Boosting (XGBoost). The Trends in International Mathematics and Science Study 2019 (TIMSS 2019) science data set was used in the study. The dataset consists of data collected from a total of 2565 students, 1309 of whom are girls (51%) and 1256 (49%) are boys. A Python-based program was used for data analysis. In the study, Area Under the Curve (AUC), accuracy, precision, recall, F1 score, Matthews correlation coefficient (MCC), and training time were used as performance indicators. The study revealed that hyperparameter tuning had a positive impact on the performance of both methods. The analysis results show that the GB method was more successful compared to the XGBoost method in all performance measures except for training time. According to the GB method, 'student confidence in science' was identified as the most influential factor in science achievement, while the XGBoost method highlighted 'home educational resources' as the most significant predictor.