Fixed effects Bayesian testing in high‐dimensional linear mixed models

Abstract Hypothesis testing for fixed effects in linear mixed model is indispensable for investigating the utility of the predictors on response. However, when the dimension of covariates exceeds the sample size, the conventional frequentist methods designed for fixed dimensions fail completely. In this article, we develop a Bayesian‐motivated test for high‐dimensional linear mixed model to examine the significance of fixed effects in group. The proposed statistic is formulated as the ratio of two quadratic forms constructed from a sequence of independent but not identically distributed random variables. The null distribution of the proposed test statistic is derived through normality approximation for quadratic forms. To facilitate the implementation of the test, we introduce an innovative one‐step iteration method to determine the critical value. Additionally, the power function under local alternatives is derived under some mild conditions. In numerical experiments, we demonstrate the power performance in comparison with the existing method and the practical utility of the proposed method.