Research on the Topic-Aware Keyword Generation Model Integrated with BERT for Social Media Short Text Analysis

Generating keywords for short social media texts is quite challenging because traditional methods are restricted by the inherent characteristics of these texts. In this study, a topic - aware keyword generation model integrating BERT (Bidirectional Encoder Representations from Transformers) is proposed. Firstly, the text is pre - processed, and character vectors are extracted via BERT. Subsequently, word vectors are generated through a specific fusion strategy and fed into the Bidirectional Gated Recurrent Unit (Bi - GRU) encoder. Meanwhile, a topic model is used to obtain the topic distribution information to assist in the learning process. In the decoding stage, the topic distribution is incorporated, and the topic attention mechanism is combined with the copy mechanism to generate keywords [1] [2]. The experiments are conducted on a Chinese general Weibo dataset, where the training set, validation set, and test set are partitioned proportionally. Accuracy, recall rate, and F1 - value are adopted as evaluation metrics. The results show that after optimizing the hyperparameters, this model outperforms traditional models such as TF - IDF and Seq2Seq in metrics like F1@1 and F1@3. The ablation experiment verifies that BERT and the topic model have a significant impact on performance improvement. Different from previous methods for handling short social media texts with limited context, the model proposed by us uniquely combines BERT and topic awareness to enhance the keyword generation performance. This model effectively alleviates the problems of feature sparsity and polysemy, improves the performance of keyword generation for short social media texts, and thus provides a reference for related research.