A Hybrid Model for Long Document of Academic Research Papers Summarization Using BiLSTM-based Sentence Classification and Hierarchical Attention Mechanism

Automatic text summarization is a key task in natural language processing (NLP).Focuses on generating concise and informative summaries from long documents while retaining their essential meaning and most relevant content.Despite advancements in deep learning, current models often struggle with long documents due to three main challenges: limited vocabulary coverage, redundant outputs, and error propagation during generation.To overcome these limitations, this paper proposes a hybrid summarization framework that combines extractive and abstractive techniques.The extractive stage uses a Bidirectional Long Short-Term Memory (BiLSTM) network along with a sentence-level classifier to score sentences.These selected sentences are then fed to the abstractive stage, where a BiLSTM-based hierarchical encoder and a decoder enhanced with a hierarchical attention mechanism are used.This architecture allows the model to capture both word-level and sentence-level dependencies, necessary for producing coherent and meaningful summaries.The presented model also solves important problems, such as the Out-Of-Vocabulary problem by using a strong embedding and tokenization technique and reducing duplication through attention-based control mechanisms.Additionally, contextual consistency during generation is enhanced to mitigate decoding errors.Experiments conducted on two benchmark scientific datasets, arXiv and PubMed, demonstrate the effectiveness of the proposed model.It achieved ROUGE-1, ROUGE-2, and ROUGE-L scores of 46.55, 22.58, and 37.11 on arXiv, and 47.94, 24.42, and 38.99 on PubMed, respectively.These results confirm the applicability of the model to scientific summarization tasks in academic and industrial NLP systems.