Influence of Performance Metrics Emphasis in Hyperparameter Tuning for Aircraft Skin Defect Detection: An Early Inspection of Weighted Average Objectives

To address the limitations of traditional aircraft skin inspection, the aviation industry and academia have increasingly been exploring the integration of computer vision technologies into the defect detection process. These implementations of computer vision technologies rely on the performance of underlying neural network models, whose effectiveness is highly influenced by their hyperparameter configuration. To obtain optimum hyperparameters, an optimization procedure is often employed to optimize a certain combination of the model’s performance metrics. However, in the aircraft skin defect detection domain, studies to inspect the effect of different emphases in the performance metrics considered in this objective function are still not widely available. In this paper, we present our early observations regarding the influence of different performance metrics’ emphases during the hyperparameter tuning process on the overall performance of a computer vision model employed for aircraft skin defect detection. In this preliminary inspection, we consider the utilization of YOLOv12 and the Bayesian Optimization approach for the defect detection model and hyperparameter optimizer, respectively. We highlight the possible performance degradation of the model after a hyperparameter tuning procedure when the weight factor distribution of the performance metrics is not carefully considered. We note several weight factors of interest that could serve as initial possible “safe spots” for further exploration.