Tail-risk-constrained portfolio optimization with cluster pre-selection and machine learning: risk-adjusted evidence from the S&P 500

Purpose Standard Gaussian risk frameworks underestimate extreme joint losses in large equity universes, exposing investors to unbudgeted drawdowns. This article develops and tests a portfolio-construction pipeline that disciplines tail risk while exploiting forward-looking return signals. Design/methodology/approach The four-stage pipeline integrates a rolling tail-index (Hill) filter, cluster pre-selection, a Conditional Value-at-Risk (CVaR) risk-parity optimizer with Generalized Autoregressive Conditional Heteroskedasticity (GARCH) and extreme value theory (EVT) marginals and a Student-t copula, and an eight-classifier machine-learning ensemble that reallocates weights within the risk budget. It is tested on S&P 500 constituents across 75 monthly walk-forward windows (January 2020–January 2026), net of transaction costs. Findings Among twelve strategies, the pipeline delivers the highest Sharpe ratio (1.106) and monthly win rate (76.0%), surpassing risk parity, Hierarchical Risk Parity and the Gaussian CVaR specification it generalizes; block-bootstrap inference confirms its advantage over passive exposure (resampled mean Sharpe 1.316). Risk parity accumulates greater terminal wealth, a trade-off, since tail constraints curb high-momentum exposure in bull markets. Research limitations/implications The analysis is limited to S&P 500 constituents and a January 2020–January 2026 evaluation period. Results may differ in other asset classes, international markets or under alternative transaction-cost assumptions. Future research could examine adaptive clustering schemes, alternative dependence structures and real-time implementation constraints. Practical implications For investors prioritizing risk-adjusted efficiency over cumulative wealth, the framework is a transparent capital-allocation tool whose tail discipline holds at roughly 97 basis points annually. Originality/value To the authors' knowledge, this is the first to combine tail-index screening, clustering, GARCH-EVT-copula CVaR optimization and a machine-learning overlay within one framework, jointly addressing the dimensionality, distributional misspecification and information-neglect problems that defeat these tools in isolation.