Rolf Sundberg's contribution to the Discussion of “On optimal linear prediction” by I. Helland

Inge Helland is to be congratulated to an impressive paper, a step in understanding PLS “vom höheren standpunkt aus”, somewhat in the spirit of “Emmy Noether's theorem” a century ago. I certainly do not pretend to have understood it all, however, so to be on safer ground in my remarks, I will stay down on earth, and mostly relate to the handling of real data. The PLS algorithm discussed is the one based on the Krylov sequence. This is the natural tool from a theoretical/population point of view, but the reader should be warned against using this one in practice, because the m $$ m $$ vectors thus generated will tend to be near collinear. An aspect avoided in the present population treatment of optimality is the “optimal” m $$ m $$ -value for the model reduction H m $$ {H}_m $$ . If there is some additional information in x $$ \mathbf{x} $$ about y $$ y $$ , outside an assumed H m $$ {H}_m $$ , then the bigger model H m ′ $$ {H}_{m^{\prime }} $$ will yield better (or at least equally good) predictions than H m $$ {H}_m $$ if m ′ > m $$ {m}^{\prime }>m $$ . In practice, however, when models are estimated from data, this need not be true, and the best m $$ m $$ -value is estimated by, for example, cross-validation (as mentioned by Helland). Principal components, PCR( m ′ $$ {m}^{\prime } $$ ), might then be better than PLS( m ′ $$ {m}^{\prime } $$ ), and perhaps about as good as PLS( m $$ m $$ ). Alternatively, if we consider m ′ < m $$ {m}^{\prime }<m $$ , say m ′ = 1 $$ {m}^{\prime }=1 $$ , ridge regression (or better the related continuum regression, CR(1)) is likely to be much better than PLS(1), also in the population set-up. The set-up in Section 6 is that the PLS model parameter θ $$ \theta $$ is assumed to have a probability distribution, implying a probability distribution for the set of regression coefficients γ j $$ {\gamma}_j $$ , for j ≤ m $$ j\le m $$ . On the other hand, the irrelevant γ j $$ {\gamma}_j $$ for j > m $$ j>m $$ are assumed fixed. This distinction between random and fixed disturbs me. Why is not all of γ $$ \gamma $$ allowed to be random? Does it matter? If β ^ a $$ {\hat{\beta}}_a $$ is close to true β $$ \beta $$ , and β ^ $$ \hat{\beta} $$ is not close to β ^ a $$ {\hat{\beta}}_a $$ , nor is β ^ $$ \hat{\beta} $$ close to true β $$ \beta $$ . If β ^ a $$ {\hat{\beta}}_a $$ is not close to true β $$ \beta $$ , this does not prevent β ^ $$ \hat{\beta} $$ from being close to true β $$ \beta $$ . This does not live up to the title of Section 7, of a condition for optimality of PLS. Finally, in Section 9, Helland parenthetically remarks: “This works well when all the x-variables are measured in the same units. When this is not the case, some scaling of the x-variables should be done first.” I want to strengthen this remark. When all x $$ x $$ are measured in the same unit, scaling is often not motivated. However, whether they are measured in the same unit or not, the choice of scaling is often more crucial than the choice between PLS and its competitors.