A Comparative Study of LSTM Efficiency vs. Transformer Power for Localized Time Series Forecasting

Forecasting multivariate time series increasingly uses deep learning, including models inspired by Neural Machine Translation (NMT).While Transformers excel at longrange dependencies, their computational overhead may not suit all problems.This study advocates the relevancy of Long Short-Term Memory (LSTM)-based encoder-decoder networks for scenarios with shorter input windows and prediction horizons.We present a comprehensive empirical evaluation of four classical LSTM-based NMT models, including variants with attention mechanisms specifically adapted for multistep time series forecasting.Our assessment focuses on their performance and the impact of varying input window sizes and prediction horizons within these computationally efficient, short-sequence contexts.We empirically compare these LSTM-based models against Transformer baselines operating under the same short input window and prediction horizon constraints.Key findings indicate that: (i) LSTM-based NMT models achieve or exceed existing state-of-the-art results for shortterm predictions; (ii) within smaller input configurations, input window size minimally affects forecasting performance for tested horizons, suggesting efficiency gains are possible; (iii) for attention-based NMT models, attention scoring critically influences accuracy, demanding careful selection; (iv) our comparative analysis demonstrates that for time series problems where immediate historical context is sufficient, LSTM-based encoder-decoders are competitive with, or even outperform, Transformer-based models while offering a more computationally efficient solution.Overall, our findings signify that original LSTM-based NMT models are robust and capable tools, particularly well-suited for short-term time series prediction tasks where local pattern capture and computational efficiency are priorities, even in the era of Transformers.