Proposing New Criteria for Early Stopping in CNN Training: A Step Towards Optimal Training

Deep neural network training, particularly with Convolutional Neural Networks (CNNs), can be resource-intensive and time-consuming. Traditional early stopping strategies, which are primarily based on validation loss, may not always accurately capture the point at which a model has learnt the most relevant features, thereby resulting in overtraining or undertraining. This work presents Singular Vector Canonical Correlation Analysis (SVCCA) as a unique method for early stopping in CNN training. SVCCA provides a dynamic method for determining when a model has converged by examining the similarity of activations’ outcomes through SVCCA. The proposed method monitors the stability of the learnt representations and indicates when additional training no longer results in significant advances in model generalization. We show that SVCCA-based early stopping can result in more efficient training by reducing overfitting and preserving or even improving model performance. Experimental results on benchmark datasets show that the proposed strategy outperforms standard early stopping methods, providing a reliable option for optimizing training time in CNNs.