Optimizing SVM hyperparameters for satellite imagery classification using metaheuristic and statistical techniques

Abstract Hyperparameters tuning is an important process through which the hyperparameters of a machine learning model are selected. The hyperparameters tuning process involves searching for the optimal hyperparameters of a machine learning model that enhances its performance and accuracy. Unlike model parameters, which are learned through the training process, hyperparameters are predefined and guide the training process. Hyperparameter tuning has gained interest due to its impact on the model’s performance. In machine learning competitions, hyperparameter tuning is often the difference between a mid-tier model and a leaderboard-topping solution. In this paper, hyperparameters of support vector machine (SVM) are optimized through the use of metaheuristic and statistical approaches to classify satellite imagery. Metaheuristic approaches are inspired by natural phenomena, and they explore the search space of the hyperparameters to find the optimal or near-optimal parameters. Statistical approaches use probabilistic models to identify the hyperparameters that maximize the model’s performance. In this paper, four metaheuristic and four statistical approaches are used. The metaheuristic approaches used are particle swarm optimization, simulated annealing, genetic algorithm, and cuckoo Search. The statistical approaches used are Bayesian optimization, successive halving algorithm, tree-structured Parzen estimator optimization, and Metropolis–Hastings Markov chain Monte Carlo. Experiments are performed on a satellite imagery classification problem. Different metrics are used to assess the performance of each algorithm, namely, accuracy, f1 score, precision, recall, and run-time. Tree-structured Parzen estimator achieves the best results for all metrics among most of the experiments with reasonable run-times proving its robustness and its practical applicability to tune the hyperparameters of SVM.