MP-UnSciBioSum: a multi-phase unsupervised document summarization method in scientific and biomedical domains

Abstract Text summarization is a crucial task in Natural Language Processing (NLP), aiming to condense lengthy documents into shorter, more compact versions while retaining the most important information. Pre-trained Language Models (PLMs) have significantly advanced automatic text summarization. However, challenges such as high training costs, input length limitations, and poor adaptability to specific domains remain. We propose a multi-phase, unsupervised approach. Initially, we segment the source document and input the segments sequentially to extend the model's text input length. Next, we utilize an unsupervised model based on T5 (Text-To-Text Transfer Transformer) to distill central textual fragments tailored for scientific and biomedical datasets. Finally, we select the most relevant sentences in the document by calculating the similarity scores between the central text fragments and the source document sentences using InfoLMScore. Our model was evaluated on datasets from the biomedical and scientific domains. Our model outperforms existing unsupervised models in terms of ROUGE-1, ROUGE-2, and ROUGE-L metrics, as well as human evaluations. It achieves superior performance and strong generalization in scientific and biomedical domains without the need for extensive labeled data and high computational resources.