Text Summarization Techniques with RNN Variant-Based Model Combining BERT and HAN

In recent years, the exponential growth of textual data has posed a significant challenge in efficiently extracting key information. In this context, automatic text summarization has emerged as a critical technique. It aims at automatically extracting core information from the text and generating a clear information overview. In this paper, we focus on the text summarization techniques, and our model is based on the Recurrent Neural Network (RNN) variant model and combines the Bidirectional Encoder Representations from Transformers (BERT) pre-training model and Hierarchical Attention Mechanism (HAN). We also discuss the related techniques of the model construction, training method, and performance evaluation. In this paper, we introduce the basic Seq2Seq model composed of RNN variants and an Attention mechanism. We also explore the advantages and feasibility of applying BERT and HAN to the text summarization tasks. We demonstrate how HAN helps enhance the RNN's attention mechanism to extract the key text information and improve the summarization capability. After constructing our BERT-RNN-HAN model, we train and test it on a professional document dataset. The experimental results show that our model, which introduces BERT and HAN, performs significantly better than the baseline model in the performance index. The summaries generated by our model are more coherent and accurate. The BERT-RNN-HAN model proposed in our paper provides useful references and innovative methods for text summarization techniques.