Multi-hypothesis prediction for portfolio optimization: A structured ensemble learning approach to risk diversification

• A new framework for portfolio allocation based on multiple hypothesis prediction and ensemble learning theory. • Theoretical and empirical validation of the link between diversity in ensemble learning and portfolio diversification. • Empirical demonstration of diversity in asset or hypothesis selection as a key source of portfolio diversification, introducing previously undiscovered limits. • Parametric control of portfolio diversification prior to decision-making by adjusting diversity in both hypothesis selection and ensemble learning stages. This work proposes a unified framework for portfolio allocation, covering both asset selection and optimization, based on a multiple-hypothesis predict-then-optimize approach. The portfolio is modeled as a structured ensemble, where each predictor corresponds to a specific asset or hypothesis. Structured ensembles formally link predictors’ diversity, captured via ensemble loss decomposition, to out-of-sample risk diversification. A structured data set of predictor output is constructed with a parametric diversity control, which influences both the training process and the diversification outcomes. This data set is used as input for a supervised ensemble model, the target portfolio of which must align with the ensemble combiner rule implied by the loss. For squared loss, the arithmetic mean applies, yielding the equal-weighted portfolio as the optimal target. For asset selection, a novel method is introduced which prioritizes assets from more diverse predictor sets, even at the expense of lower average predicted returns, through a diversity-quality trade-off. This form of diversity is applied before the portfolio optimization stage and is compatible with a wide range of allocation techniques. Experiments conducted on the full S&P 500 universe and a data set of 1,300 global bonds of various types over more than two decades validate the theoretical framework. Results show that both sources of diversity effectively extend the boundaries of achievable portfolio diversification, delivering strong performance across both one-step and multi-step allocation tasks.