Fluctuations of sample eigenvalues in a spiked covariance model with Haar-rotated spikes: a transition at $p^2 \asymp n$
We study the outlying eigenvalues of a sample covariance matrix whose population covariance is itself random. The population covariance is $Σ_p=A_p+O_pB_pO_p^*+σ^2I_p$, where $A_p$ and $B_p$ are deterministic positive semidefinite matrices of fixed ranks $r$ and $s$, and where $O_p$ is Haar distributed on the orthogonal group in the real case, and on the unitary group in the complex case. Conditionally on $O_p$, we observe $n$ independent centered Gaussian vectors with covariance $Σ_p$, and we build the sample covariance matrix $\widehatΣ_p$ from these vectors. The fluctuations of the spike eigenvalues of $\widehatΣ_p$ come from two different sources: the Haar matrix $O_p$, which displaces the population spikes at the scale $1/p$, and the sampling, which acts at the scale $n^{-1/2}$. Comparing these two scales amounts to comparing $p^2$ with $n$, so the ratio which governs the fluctuations is $p^2/n$, while in the classical spiked model it is the aspect ratio $p/n$. We give the limiting law of each spike in four regimes where $n,p\to \infty$. When $p/n\to c>0$, and when $p/n\to0$ with $p^2/n\to\infty$, the sampling term dominates, the limit does not depend on $O_p$ and is Gaussian, as in the classical spiked model. When $p^2/n\to0$, the Haar randomness dominates, and the limit is a weighted sum of independent $χ^2_1$ variables in the real case, and of independent exponential variables in the complex case. When $p^2/n\toτ\in(0,\infty)$, the two terms are of the same order, and the limit is the convolution of Gaussian and $χ^2_1$ or Exponential variables. We then apply these results to a linear mixed model with a continuous batch effect, in which the Haar assumption is what makes the separation of the signal and of the batch possible, and we build a test for the value of a single spike.
Authors
- Noriyoshi Sakuma (ORCID: https://orcid.org/0000-0001-7309-4106)
- Jean-Lou Valeau
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-11-12
- DOI
- https://doi.org/10.48550/arxiv.2611.16739
- Primary Topic
- Random Matrices and Applications
- Type
- preprint