Probabilistic Forecasting and Robust Optimization for Managing Uncertainty in Smart Beta Portfolio Optimization

ObjectiveThis study employs a probabilistic forecasting approach and robust optimization to address parameter uncertainty in portfolio optimization models within the Iranian capital market. The main focus is on enhancing portfolio performance by accounting for uncertainty and utilizing machine learning models to construct portfolios with maximum Sharpe ratios. MethodsTwo common approaches are applied to incorporate parameter uncertainty into the portfolio optimization model. The first approach is robust optimization, which defines an uncertainty set for each parameter and analyzes the problem in such a way that the solution remains optimal even under worst-case parameter realizations. The second approach involves an advanced machine learning model, Natural Gradient Boosting (NGBoost), whose outputs were employed within a probabilistic forecasting framework. The model inputs included five technical indicators: Relative Strength Index (RSI), Moving Average Convergence/Divergence (MACD), Average True Range (ATR), Average Price Trading (ATP), and Momentum. Technical analysis is one of the main approaches in examining and forecasting financial market trends, which is based on the study and evaluation of historical price and trading volume data. This method assumes that all fundamental and psychological information of the market is reflected in prices, and that price movements form recognizable and repeatable patterns. The study is conducted across 10 industries, including basic metals, oil refining, banking and financial institutions, petrochemicals and chemicals, automotive, cement, pharmaceuticals, precious metals, rubber and plastics, and metallic minerals. The aforementioned industries are among the largest sectors of the Iranian capital market and, in terms of market value, constitute a substantial portion of the market. These industries encompass a wide range of production and service domains, each playing a fundamental role in the country’s economy and industrial development. Overall, the synergy of these industries strengthens economic diversification, foreign exchange earnings, employment, and sustainable development. After applying robust and probabilistic forecasting models in portfolio optimization, the results were compared against two benchmark portfolios—an equal-weight portfolio and the Markowitz mean-variance model—using the Sharpe ratio as the evaluation metric. Results"Using data from March 2022 to March 2024 for training the NGBoost model and estimating parameters for robust optimization, and 2024 data as the test set, portfolios were constructed for all ten industries. Their out-of-sample risk and return were then calculated. The comparison indicated that both proposed approaches significantly outperformed the benchmark portfolios, achieving higher Sharpe ratios at the 99% confidence level. ConclusionThe findings demonstrate that employing distributional rather than point forecasts, combined with smart beta strategies and robust parameter consideration in portfolio optimization, leads to portfolios with superior risk-return trade-offs. This enhanced performance is statistically significant at the 99% level. Furthermore, the results indicate that incorporating technical indicators as explanatory factors for returns can effectively improve return predictability. Leveraging these indicators in smart beta portfolio construction yields portfolios with superior performance.