Isotonic and Convex Regression: A Review of Theory, Algorithms, and Applications

Shape-restricted regression provides a flexible framework for estimating an unknown relationship between input variables and a response when little is known about the functional form, but qualitative structural information is available. In many practical settings, it is natural to assume that the response changes in a systematic way as inputs increase, such as increasing, decreasing, or exhibiting diminishing returns. Isotonic regression incorporates monotonicity constraints, requiring the estimated function to be nondecreasing with respect to its inputs, while convex regression imposes convexity constraints, capturing relationships with increasing or decreasing marginal effects. These shape constraints arise naturally in a wide range of applications, including economics, operations research, and modern data-driven decision systems, where they improve interpretability, stability, and robustness without relying on parametric model assumptions or tuning parameters. This review focuses on isotonic and convex regression as two fundamental examples of shape-restricted regression. We survey their theoretical properties, computational formulations based on optimization, efficient algorithms, and practical applications, and we discuss key challenges such as non-smoothness, boundary overfitting, and scalability. Finally, we outline open problems and directions for future research.