Hybridizing GARCH and Random Forest for Enhanced Modeling of Volatility Clustering

Clustered volatility refers to a non-stationary process in which the mean and variance of a time series change over time rather than remaining constant, reflecting the dynamic nature of volatility, which fluctuates in response to varying market or structural conditions and often results in alternating periods of high and low volatility. It can be identified through visual inspection of the time series and formal autocorrelation tests using the Autocorrelation Function, where significant correlations at multiple lags indicate that current volatility depends on past volatility, meaning that large or small past movements are likely to be followed by similarly large or small movements regardless of direction. This study investigates the enhancement of the Autoregressive Conditional Heteroskedasticity (ARCH) model and its generalization, the GARCH model, through integration with Random Forest (RF) machine learning techniques. Using daily historical stock prices of Apple Inc. from January 2020 to July 2025, focusing on the closing price, four models were evaluated: ARCH, GARCH, and their Random Forest–enhanced versions. Traditional models showed weak performance, with ARCH producing the highest errors and GARCH performing slightly better, while hybrid models significantly improved results. GARCH with Random Forest explained approximately 49.3% of the variance, whereas ARCH with Random Forest achieved the best performance, with very low errors and explanatory power of 97.82%. These findings indicate that traditional models have limited predictive capability. In contrast, hybrid models provide more accurate, adaptive, and dynamic forecasts, highlighting the potential of combining econometric models with machine learning to enhance the precision of volatility prediction in financial markets.