Double descent in high-dimensional linear discriminant analysis

The double descent phenomenon observed in deep neural networks has called into question conventional perspectives on model complexity in machine learning, particularly the classical U-shaped bias–variance tradeoff curve. While this phenomenon has been extensively studied in modern high-capacity models, relatively little attention has been paid to its implications for classical statistical methods such as linear discriminant analysis (LDA), a long-standing classification technique that seeks a linear combination of features that best separates two classes. We leverage random matrix theory to characterize the asymptotic misclassification error of LDA as the dimension p and sample size n grow at the same rate p/n→γ. Our main contribution is an asymptotic limit for the LDA risk: for a general population covariance matrix when γ∈(0,1), and for the pseudo-inverse classifier with isotropic covariance when γ∈(1,∞). Together, these give an explicit form for the full double-descent risk curve of LDA in both the under- and over-parameterized regimes. We validate our theoretical findings through simulation studies and an empirical analysis of the ARCENE cancer classification dataset.