Performance Evalution of Hyperparameter Tuning Techniques for Machine Learning Models in Spam Detection

Hyperparameter selection plays a pivotal role in optimizing the performance of machine learning models, particularly for tasks such as spam detection, where both accuracy and computational efficiency are critical. In this study, the Spambase dataset from the UCI Machine Learning Repository was used to evaluate six machine learning models: Multi-Layer Perceptron (MLP), Light Gradient Boosting Machine (LightGBM), Random Forest (RF), Extreme Gradient Boosting (XGBoost), Decision Tree (DT), and K-Nearest Neighbors (KNN). These models were optimized using six hyperparameter optimization techniques: Covariance Matrix Adaptation Evolution Strategy (CMA-ES), Differential Evolution (DE), Bayesian Optimization (BO), Genetic Algorithm (GA), Tree-structured Parzen Estimator (TPE), and Particle Swarm Optimization (PSO). The analysis highlights the significant impact of model and optimization method selection on predictive performance and resource efficiency. Based on the experimental results, XGBoost achieved the highest accuracy (0.9824), showcasing its effectiveness in spam detection tasks. LightGBM demonstrated a favorable balance between accuracy (0.9674) and optimization speed, making it a practical alternative. Decision Tree models were notable for their computational efficiency, optimizing in as little as 2.81 seconds with TPE. Bayesian Optimization and TPE emerged as the most efficient hyperparameter tuning methods, achieving competitive accuracy with minimal time costs. Future studies could focus on addressing challenges such as computational complexity and evolving spam patterns by exploring advanced optimization strategies and adaptive deep learning models.