A novel transformer-based dual attention architecture for the prediction of financial time series

Financial prediction has gained significant attention due to the complex and non-linear dynamics of the market. A promising approach for generating accurate predictions is Transformers. Encoder-decoder structures efficiently capture complex temporal dependencies and patterns within large-scale data. However, relying on a single attention mechanism may limit the model’s ability to capture more intricate relationships. This paper proposes a dual attention architecture to improve the encoder-decoder framework for financial forecasting. First, the Price Attention Network (PAN) extracts complex features from price data and forecasts future prices using historical price inputs. Two key improvements are introduced to enhance self-attention: a Masked Self-Attention module focusing on the most relevant information and Multi-head Attention facilitating more profound insights into the data. Second, the Nonprice Attention Network (NAN) is proposed as a parallel network that processes related financial features. This network utilizes ConvLSTM, BiGRU, and Self-Attention to dynamically weigh and extract meaningful information from nonprice data. Finally, the PAN and NAN networks are integrated, enhancing prediction accuracy. The proposed approach outperforms five state-of-the-art models. Moreover, qualitative assessments of over 26 financial datasets, spanning large and small datasets with short and long histories, further validate the proposed model's ability. Evaluations using seven metrics show the model’s superiority, achieving a Mean Absolute Error (MAE) of 0.01991, Mean Squared Error (MSE) of 0.00084, Mean Pinball Loss (MPL) of 0.00996, Symmetric Mean Absolute Percentage Error (SMAPE) of 3.03324, and Mean Absolute Scaled Error (MASE) of 1.85436. This framework represents a significant advancement in financial prediction, offering accurate and interpretable forecasts across various time series tasks.