Optimizing ML and DNN by Genetic Algorithms

This study investigates the application of genetic algorithms (GA) for hyperparameter optimization in machine learning models, including Logistic Regression, Random For-est, Support Vector Machine (SVM), XGBoost, and Multilayer Perceptron (MLP). The performance of models with default settings, optimized using grid search, and optimized using GA is compared using the F1 macro score on the “Heart Disease” dataset from the UCI Machine Learning Repository. The results demonstrate that hyperparameter optimization, both through grid search and GA, generally improves model performance compared to default configurations. GA optimization consistently outperforms or matches grid search performance for all models tested. For GA-optimized MLP, a specialized initialization mechanism, crossover, and mutation operators are introduced to optimize both network structure and hyperparameters. The best GA-optimized MLP achieves the highest F1 macro score of 87% among all experiments, surpassing both default and grid search optimized models. The study highlights the potential of GA for automating hyperparameter tuning, efficiently exploring the search space, and finding near-optimal configurations. As the complexity of machine learning models grows, evolutionary algorithms like GA are likely to become increasingly important for hyperparameter optimization in the field of machine learning.