A Systematic Approach of Hyper-Parameter Optimization to Improve Neural Network Model Performance

This study investigates the effectiveness of different hyper-parameter optimization methods for a neural network model predicting solar array voltage peak <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{V}_{\mathrm{m}\mathrm{p}}$</tex>. The performance of four methods is compared: Trial and Error, GS, RS, and PSO. Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and R-squared were used to evaluate model performance. PSO achieved the best performance (MAE = 4.52, MSE = 90.63, RMSE = 9.52, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{R}^{2}=0.81$</tex>) compared to other methods. PSO also achieved the fastest search time (60.46 second) compared to other methods. This study highlights PSO's potential as an effective hyper-parameter optimization algorithm, producing to a model with superior performance metrics. This study also explores the impact of varying search space dimensions and swarm population on model performance, emphasizing the importance of selecting an appropriate population size, and investigates the effects of adjusting PSO cognitive and social parameters, highlighting the necessity of balancing these parameters for optimal PSO performance.