Neural Networks in Financial Engineering: Forecasting Using RNN, LSTM and Hybrid Models in Varying Volatilities

Financial engineering is experiencing growing complexity and volatility in financial markets, particularly in cryptocurrency and equity indices.Effective forecasting needs advanced predictive tools.This study compares the performance of RNN, LSTM and hybrid models (GARCH-LSTM and CNN-LSTM) in two varying volatility data, low volatility S&P 500 and high volatility bitcoin crypto.Time series data from November 2019 to November 2024 was used.The four-model forecast performance was evaluated using multiple metrics MSE, MAE, RMSE, Huber Loss, and Log-Cosh Loss.In all performance measurement metrics, hybrid models (CNN-LSTM and GARCH-LSTM) outperformed the individual models (LSTM and RNN).GARCH-LSTM was found superior in forecasting highly volatile bitcoin (BTC/USD) prices (MSE: 224,023; MAE: 370.70); while CNN-LSTM was found superior in forecasting less volatile S&P 500 series (MSE: 10,662; MAE: 79.72).LSTMs outperform RNNs in both low volatility (S&P 500) as well as high volatility data (BTC/USD).Hybrid models are considered superior in time series forecasting.