On the Foundational Arguments of Sufficient Dimension Reduction

ABSTRACT Sufficient dimension reduction (SDR) refers to supervised methods of dimension reduction that apply in the context of regression, interpreted broadly. SDR started in the early 1990's with methodology to reduce linearly the predictor dimension without loss of information about the conditional distribution of the response given the predictors. The field grew quickly and today it is vast. The early ideas and methods have been formalized, extended, specialized and adapted to many problems in statistics. A comprehensive synopsis of everything covered by SDR would be truly substantial. Instead, the focus of this overview is on the ideas and philosophy of SDR. While the field is vast, there are a few foundational ideas that define the area generally and have been adapted to different problems. In this overview, we elucidate these ideas by explaining the historical ambience, exploring the genesis of the ideas and discussing how they are adapted for various problems. There is also literature on ‘dimensionality reduction’ that is beyond the scope of this overview. Examples include uniform manifold approximation and projection, neural PCA, kernel PCA, and locally linear embedding. Dimension reduction of data that are not meaningfully stochastic is also outside the scope of this article. The worlds of dimensionality reduction and sufficient dimension reduction were largely developed independently, even when in retrospect the developments involve overlap.