From screening to variable selection by an iterative nonparametric procedure based on derivatives

Abstract To identify the true relevant predictors in a nonparametric regression model with ultra-high-dimensional data, we propose a general iterative procedure that is able to transform a screening procedure into a variable selection one, without requiring the number of true relevant variables to be finite. This fully nonparametric procedure is based on a novel combination of DELSIS (a model-free variable screening method based on empirical likelihood and derivative estimation) and PenGAM (a penalised variable selection method based on splines). The main advantage of the new proposal is its robustness to the presence of correlation among the predictors and its capability to correctly identify the whole set of relevant covariates, including those marginally uncorrelated but jointly related to the response (“hidden covariates”) and those with low signal (“weak covariates”), much better than the main alternative approaches. From a theoretical point of view, we show the consistency of the proposal and its faster estimation rate with respect to the classical penalised approaches. Finally, we illustrate the performance of the new procedure through some simulations and an empirical analysis.