Enhancing Time Series Prediction in Industrial Processes: A Self-Attention BiLSTM Approach

The development of data-driven soft sensors for modeling complex data, particularly in scenarios characterized by strong nonlinearity, high dimensionality, cross-correlation and autocorrelation, remains a significant challenge in process data analysis. Bidirectional Long Short-Term Memory (BiLSTM) networks are highly effective in capturing both forward and backward dependencies within sequences, thereby enhancing the extraction of temporal features from time series data. However, BiLSTM networks may struggle to capture global dependencies, especially in long sequences. To address this limitation, the self-attention mechanism, which excels in modeling global relationships across all positions in the input sequence, can be integrated with BiLSTM by applying weighted attention to the two different directions of BiLSTM, thereby enhancing its performance. This study proposes a novel method called SA-BiLSTM, which combines the strengths of the BiLSTM network and the self- attention mechanism. The self-attention layer captures global dependencies, while the BiLSTM layer extracts comprehensive temporal features in both directions, which are then passed to a fully connected layer for final prediction. Extensive comparisons with traditional methods such as BiLSTM, LSTM with added self-attention mechanism, LSTM, RNN, and SVM validate the superior performance of the proposed SA-BiLSTM method in handling complex time series data.