Parallel Hybrid LSTM with Longitudinal Memory: To Handle Longer Sequential Data

Long Short-Term Memory (LSTM), a variant of Recurrent Neural Network (RNN), is designed to process and predict sequential information effectively, addressing issues such as vanishing and exploding gradients. However, the prolonged training time of LSTM has led to the development of alternative architectures enabling parallel data processing, which accelerates convergence. Despite these advancements, utilizing parallel architectures for extensive sequential data has reintroduced a challenge related to vanishing gradient, prompting ongoing research to overcome this limitation. To mitigate data loss caused by these setbacks, this paper proposes an external memory mechanism to enhance the handling of long-term sequences. An attention mechanism is incorporated, along with a gate designed to assign significance to critical data for future implementation. The proposed architecture is trained and tested on the IMDB dataset, highlighting performance improvements compared to LSTM-based parallel architectures without a memory unit and other existing technologies. Experimental results demonstrate that the proposed model achieves notable advancements over earlier techniques.