An innovative approach to stock forecasting: combining LSTM, VMD, and artificial rabbit optimization

Abstract Stock price time series exhibit inherent nonlinearity, nonstationarity, and volatility, which limit the predictive capacity of classical econometric models and conventional machine learning approaches. To address these challenges, this study proposes a novel hybrid framework, LSTM-ARO-VMD, that integrates Variational Mode Decomposition (VMD), Long Short-Term Memory (LSTM) networks, and the Artificial Rabbits Optimization (ARO) algorithm within a unified, simultaneously optimized architecture. Unlike prior approaches that treat decomposition and deep learning optimization as independent steps, the proposed framework jointly optimizes the VMD parameters (number of modes K and penalty factor α ) alongside the LSTM hyperparameters using ARO, enabling adaptive multi-scale feature extraction from non-stationary financial signals. The model was evaluated on daily closing price data of eight stocks across NSE, NASDAQ, and NYSE, namely ORCL, MSFT, HAL, GS, CTSH, BAC, and AMZN, covering the period 2009–2025 (2015–2025 for SBIN). Performance was assessed using RMSE, MAE, MAPE, NRMSE, and a Combined Metric (CM). The proposed model achieved MAPE values as low as 0.0263% (AMZN), 0.0119% (BAC), 0.0094% (CTSH), and 0.0691% (HAL), with RMSE values of 0.0316, 0.0101, 0.0021, and 0.0013, respectively. Against five baselines LSTM, LSTM-PSO, LSTM-SCA, LSTM-ARO-EMD, and LSTM-ARIMA-GARCH, the proposed model achieved an improvement in RMSE on average. These results demonstrate that integrating decomposition, deep learning, and metaheuristic optimization into a unified framework provides a scalable approach for stock price forecasting.