Adaptive Hyperparameter Optimization Strategies for Neural Network Models in Engineering Design Optimization

Surrogate-based optimization using neural networks (NNs) reduces computational costs in engineering design but depends heavily on systematic hyperparameter optimization (HPO). This study compares HPO methods—including grid search, random search, Bayesian optimization (BO), hyperband (HB), and BOHB—using analytical functions and aerodynamic shape optimization (ASO). The study first explores HPO in a one-shot method where NNs are trained with datasets of different sample sizes. The analysis then extends to adaptive HPO approaches within an efficient global optimization (EGO) framework using NNs, which employs sequential sampling. Under this framework, static HPO (maintaining initially optimized HPs), periodic HPO (adjusting HPs every five infill points), and dynamic HPO (adjusting HPs after each infill point) are compared. In ASO with a one-shot method, BOHB achieves a drag coefficient ([Formula: see text]) of 117 drag counts (d.c.)—close to BO’s 115 d.c.—while requiring only 30.5% of BO’s computational time with 500 samples. Additionally, in ASO with sequential sampling, periodic HPO effectively balances performance and computational efficiency by achieving a [Formula: see text] of 119 d.c. with 25 initial samples and 50 infills. Dynamic HPO reduces [Formula: see text] to 113 d.c. but at a higher cost compared to periodic HPO, which offers a balance between drag reduction and computational expense.