Quantifying overfitting in deep learning

Overfitting is a persistent challenge in deep learning (DL) that greatly affects model generalization and performance. Traditional methods for assessing overfitting often lack practical applicability in real-world DL scenarios, resulting in ambiguous quantification and complicating the selection of effective mitigation strategies. This study introduces the Overfitting Risk Inequality (ORI), a novel approach that quantifies overfitting by measuring the Wasserstein distances between actual training/validation loss curves and an ideal training loss curve. Our results demonstrate the effectiveness of ORI across diverse benchmark DL models, including Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Transformers (such as Vision Transformers), and Long Short-Term Memory networks (LSTMs), in both binary and multiclass classification tasks. These findings highlight ORI’s sensitivity and accuracy in quantifying overfitting, offering a more precise and versatile tool for evaluating DL models in practical applications.