A non‐asymptotic analysis of the single component PLS regression

ABSTRACT This paper investigates some theoretical properties of the Partial Least Squares method. We focus our attention on the single‐component case, which provides a useful framework to understand the underlying mechanism. We provide a non‐asymptotic upper bound on the quadratic loss in prediction with high probability in a high‐dimensional regression context. The bound is attained thanks to a preliminary regularization on the first PLS component. In a second time, we extend these results to the sparse Partial Least Squares approach. In particular, we exhibit upper bounds similar to those obtained with the lasso algorithm, up to an additional restricted eigenvalue constraint on the design matrix.