Estimation of the Generalization Ability of Deep Learning Model Based on Complexity

With the continual accumulation of data and advances in machine learning technology, it has become feasible to construct generalized estimation models based on sets of indicators. Viewing complexity indicators as independent variables and model generalizability as the dependent variable, the scenario transforms into a regression problem. Hence, this chapter examines the application of regression prediction models grounded in machine learning for the purpose of estimating the generalizability of deep convolutional networks. Given the construction risks present in generalizability measurements toward test datasets, this study proposes a generalizability measurement method that relies solely on the intrinsic attributes of the model. Several common machine learning models-SVR, RF, XGBoost, and FCNN-are employed to establish regression models between model complexity and generalization gap. To address the difficulty of manually designing hyperparameters in machine learning models, Bayesian optimization is utilized for hyperparameter tuning. The regression effectiveness of the four machine learning models, post hyperparameter optimization, is analyzed experimentally, selecting the model with the best performance as the generalizability estimation model for this study. The results indicate that the BO-XGBoost generalizability estimation model achieved the best regression outcomes, with R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> values of 0.994 and 0.997 in two model sets respectively.