Improving Text Summarization with Filtered Attention Mechanism on BERT

This paper presents FAM-BERT, a BERT-based text summarization model enhanced with a Filtered Attention Mechanism (FAM), addressing challenges in content selection and redundancy in abstractive summarization. By integrating FAM, the model dynamically identifies and focuses on key content while suppressing irrelevant details, resulting in concise and coherent summaries. Extensive experiments on the CNN/Daily Mail dataset demonstrate that FAM-BERT achieves state-of-the-art performance, surpassing baseline models such as Transformer, PGN, and BERT across ROUGE-1, ROUGE-2, and ROUGE-L metrics. These results validate the efficacy of FAMBERT in improving content coverage, fluency, and structural consistency, offering a robust solution for high-quality abstractive text summarization.