R. D. Cook and L. Forzani's contribution to the Discussion of ‘On optimal linear prediction’ by I. Helland

We congratulate Inge on his intriguing article. He has long been known for his work on PLS regression and we have followed his recent venture into quantum mechanics. Pursuing a syntheses of Statistics, PLS, and quantum mechanics is a laudable goal and we wish him future success. In this brief comment, we focus on aspects of PLS: its structure as a dimension reduction (DR) method, its operational characteristics in non-linear regressions and insights into its optimality. Notation, references, and context are the same as those used by Helland, unless stated otherwise. PLS regression is a sufficient dimension reduction (SDR) method, and thus has SIR, SAVE, PhD, …as its cousins. PLS algorithms such as NIPALS and SIMPLS perform DR and then automatically fit a linear model based on the reduced predictors. The DR step returns a moment-based estimate of a basis for ℰ Σ x x ( ℬ ) $$ {\mathcal{E}}_{{\boldsymbol{\Sigma}}_{\mathbf{xx}}}\left(\mathcal{B}\right) $$ , the Σ x x $$ {\boldsymbol{\Sigma}}_{\mathbf{xx}} $$ -envelope of ℬ : = span ( β ) $$ \mathcal{B}:= \mathrm{span}\left(\boldsymbol{\beta} \right) $$ , defined as the smallest reducing subspace of Σ x x $$ {\boldsymbol{\Sigma}}_{\mathbf{xx}} $$ that contains ℬ $$ \mathcal{B} $$ . The dimension of envelope is the true number of components, m $$ m $$ . Under the linear model, y ╨ x | P ℰ Σ x x ( ℬ ) x , where P ( · ) $$ {P}_{\left(\cdotp \right)} $$ projects onto the indicated subspace. Thus, the envelope is a DR subspace. SDR methods have a long history of successful application, particularly when n ≫ p $$ n\gg p $$ . Cook and Forzani (2024, Section 4.3.2) gave an example where PLS dominates OLS in a regression with p = 4 $$ p=4 $$ and n = 79 $$ n=79 $$ . Generally, we do not associate PLS with just large p $$ p $$ regressions, but view it as an SDR method that may be effective when p $$ p $$ is large or small, even with p = 4 $$ p=4 $$ . Helland expands the foundations of PLS by establishing certain optimality conditions in the context of linear regression. PLS foundations can be expanded also by considering its scope. In particular, the DR step of a PLS algorithm can be serviceable in non-linear regression, as it is in linear regression. Let 𝒮 E ( y | x ) denote the central mean subspace for the regression of y $$ y $$ on x $$ \mathbf{x} $$ , defined as the intersection of all subspaces 𝒮 so that y ╨ E ( y | x ) | P 𝒮 x . Then under mild conditions, ℰ Σ x x ( ℬ ) ⊆ 𝒮 E ( y | x ) (Cook & Forzani, 2021). If the regression follows a single index model, 𝒮 E ( y | x ) = f ( β T x ) then ℰ Σ x x ( ℬ ) = 𝒮 E ( y | x ) . This implies that the DR arm of a PLS algorithm can be used to study the model y = E ( y | x ) + ε $$ y=E\left(y|\mathbf{x}\right)+\varepsilon $$ graphically, without necessarily specifying E ( y | x ) $$ E\left(y|\mathbf{x}\right) $$ . It seems that Theorems 3, 4, and 5 were constructed as stepping stones, leading inescapably to Theorem 6 which contains condition (19) for the optimality of PLS-like methods. While theoretically informative, it's not clear how to use (19) for guidance in practice. Let β ^ a $$ {\hat{\boldsymbol{\beta}}}_a $$ denote the PLS estimator of β $$ \boldsymbol{\beta} $$ with a $$ a $$ components, so β ^ p $$ {\hat{\boldsymbol{\beta}}}_p $$ denotes the OLS estimator. We conclude with a question: Envelopes are invariant or equi-variant under certain classes of predictor transformations. Is the group defined by Helland related in any material way to this variant structure of envelopes? The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.