Does Outcome Measurement Matter for Forecasting? Evidence from Fiscal Vulnerability in Developing Economies

Abstract The construction of multidimensional outcome measures can influence the evaluation and interpretation of forecasting models, yet this sensitivity is often overlooked in empirical applications. This study examines how alternative definitions of fiscal vulnerability affect predictive performance, incremental forecasting value beyond persistence, and surveillance conclusions in developing economies. Using an annual panel of 127 developing economies over 1982–2024, three vulnerability measures are constructed from eight common fiscal, macroeconomic, and external indicators, capturing the breadth of adverse conditions, the severity of threshold exceedances, and a principal component-based measure. Forecasting models are assessed using chronological validation and out-of-sample evaluation against logistic regression and fitted persistence benchmarks. In the 2018–2023 external evaluation, the selected model achieves AUCs of 0.819, 0.813, and 0.959 for breadth, severity, and PCA, respectively. However, incremental AUC gains over persistence are statistically significant only for breadth (+ 0.022) and severity (+ 0.020), with no corresponding improvement established for PCA. Permutation importance identifies fiscal balance and government debt as leading predictors, while the contributions of other indicators differ across definitions. Breadth-based forecasts also provide additional information about new vulnerability onsets, and country-level surveillance rankings vary across measures. These findings demonstrate that outcome measurement affects conclusions about forecasting performance, predictor relevance, and surveillance priorities, while high apparent predictability does not necessarily translate into incremental predictive value. JEL classification: C45; C53; E62; H63; O19