A GRU-SAPD Neural Network for Short-Text Sentiment Classification

Deep learning models and attention mechanisms have been widely applied to short-text sentiment classification tasks, achieving remarkable performance. Particularly, gated recurrent units (GRUs) excel in sequence modeling of sentiment texts and can further enhance the accuracy of short-text sentiment classification by extracting more important features through attention mechanisms. However, traditional attention mechanisms assign different weights to the hidden states encoded by the encoder, overlooking the emotional tendencies brought by the probability distribution differences across different categories. To address these issues, this paper proposes a novel model architecture gated recurrent unit self-attention probability distribution called GRU-SAPD, which includes GRU, self-attention mechanism, and a probability distribution module. The probability distribution block, determined before model training, performs re-weighting on the outputs weighted by self-attention. In the GRU-SAPD model, GRU extracts long-term dependencies between text sequences through word embedding vectors, then self-attention mechanism weights the extracted feature relations, and finally, the probability distribution block enhances the features based on the probability distribution weights. Subsequently, a fully connected layer is used to classify the extracted contextual sentiment features. Experiments were conducted on one spam message classification dataset and three short-text sentiment classification datasets, and GRU-SAPD was thoroughly analyzed. Experimental results demonstrate that GRU-SAPD outperforms other state-of-the-art short-text sentiment classification methods in terms of classification accuracy.