Random Matrix Theory Predictions of Dominant Mode Rejection SINR Loss due to Signal in the Training Data

Detection and estimation performance depends on signal-to-interference-plus-noise ratio (SINR) at the output of an array. The Capon beamformer (BF) designed with ensemble statistics achieves the optimum SINR in stationary environments. Adaptive BFs compute their weights using the sample covariance matrix (SCM) obtained from snapshots, i.e., training samples. SINR loss, the ratio of adaptive to optimal SINR, quantifies the number of snapshots required to achieve a desired average level of performance. For adaptive Capon BFs that invert the full SCM, Reed et al. derived the SINR loss distribution and Miller quantified how the desired signal's presence in the snapshots degrades that loss. Abraham and Owsley designed dominant mode rejection (DMR) for cases where the number of snapshots is less than or approximately equal to the number of sensors. DMR's success in snapshot-starved passive sonar scenarios led to its application in other areas such as hyperspectral sensing and medical imaging. DMR forms a modified SCM as a weighted combination of the identity matrix and the dominant eigensubspace containing the loud interferers, thereby eliminating the inverse of the poorly estimated noise subspace. This work leverages recent random matrix theory (RMT) results to develop DMR performance predictions under the assumption that the desired signal is contained in the training data. Using white noise gain and interference suppression predictions, the paper derives a lower bound on DMR's average SINR loss and confirms its accuracy using Monte Carlo simulations. Moreover, this paper creates a new eigensubspace leakage estimator applicable to broader RMT applications.