Enhancing CAPE Models: Leveraging Conditional Clustering-Based Linear Regression to Predict 10-Year US Equity Returns

A forecast of long-term equity returns plays a crucial role in portfolio construction and asset allocation strategies. One of the leading indicators to assess stock market valuations was the cyclically adjusted price-to-earnings (CAPE) ratio, developed by Professors John Campbell and Robert Shiller in 1988. The CAPE model, however, has lost its predictive accuracy since 1985 because of its reliance on the volatile beta coefficient. This article presents an enhanced methodology that improves the CAPE model with regime analysis using clustering to improve the modeling dynamic relationship of CAPE with long-term equity returns. This methodology delivers most of the benefits of simplicity and wide applicability that the original CAPE model had but allows new insight into regime-specific dynamics that can be matched with other macroeconomic factors to add value to this analysis. This research therefore yields very promising results and may provide a more reliable, richer analytical insight.