Long short-term memory networks in learning memory inconsistencies of stock markets

Abstract Deep learning enables neural networks to improve prediction performance through data supplementation. In financial time series forecasting, however, such data-driven approaches can encounter limitations where additional data degrade performance, contrary to common expectations. While more data can still be beneficial, it may introduce systemic concept drift due to the complex nonstationarities of stock price index time series, thereby exacerbating overfitting. One such drift is memory inconsistency: locally measured long memories fluctuate over time, alternately approaching and deviating from the random walk condition. We address this problem by typifying memory inconsistencies into two simplified forms: long-term dependent-to-independent (D2I) and long-term independent-to-dependent (I2D) inconsistencies. The first experiment, which uses U.S. stock price indices, suggests that additional training examples may lead to performance deterioration of long short-term memory (LSTM) networks, especially when memory inconsistencies are prominent. Since stock markets are influenced by numerous unknown dynamics, the second experiment, which uses simulated mean-reverting time series derived from the fractional Ornstein–Uhlenbeck (fOU) process, is conducted to focus solely on challenges arising from memory inconsistencies. The experimental results demonstrate that memory inconsistencies disrupt the performance of LSTM networks. Theoretically, additional errors from D2I and I2D inconsistencies increase as the time lag increases. Since LSTM networks are inherently recurrent, causing information from distant steps to attenuate, they fail to effectively capture memory inconsistencies in practical offline learning schemes. Nonetheless, transplanting pretrained memory-consistent gate parameters into the LSTM model partially mitigates the performance deterioration caused by memory inconsistencies, suggesting that memory augmentation strategies have the potential to overcome this problem. As such a memory augmentation method, we propose the Gate-of-Gates (GoG) model, which extends the capacity of LSTM gates and demonstrates that it can mitigate additional errors arising from memory inconsistencies.