Statistical Performance and Economic Value in Machine-Learning-Based Financial Forecasting

Though machine learning has been extensively used in financial forecasting, previous literature mainly focuses on measuring model performance based on statistical measures such as R² and RMSE without paying attention to whether those models create any economic benefit. In order to explore whether the improvement in statistical measures creates any economic benefit, this paper establishes a methodology of converting statistical measures into economic values and then performs a comparative analysis on four typical papers that report both kinds of measures. Three patterns about the relation between statistical measures and economic values are found from the analysis, which are positive correlation, negative correlation, and conditional correlation. Although statistical accuracy is a prerequisite for creating economic value, it is insufficient since the transformation from statistical accuracy to economic value faces several restrictions, including over-fitting, signal degradation, and mismatch of performance measures. The findings suggest that investors should not rely solely on statistical metrics for model selection, but rather incorporate risk-adjusted performance measures such as the Sharpe ratio and maximum drawdown, while also accounting for model complexity and signal persistence when constructing trading strategies.