Data Pyramid for Enterprise News Summarization: A Three-Stage Knowledge-Enhanced Approach

Enterprise news summarization faces significant challenges due to the limited availability of labeled data, hindering the performance of traditional summarization models. This paper introduces a novel Knowledge-Enhanced T5-Pegasus model specifically designed to address this issue. Our approach employs a three-stage progressive fine-tuning strategy: (1) pre-training on large-scale general text data; (2) domain-adaptive fine-tuning on a corpus of enterprise news, incorporating a novel adaptive keyword attention mechanism to capture crucial domain-specific information; (3) contrastive learning on the target small dataset, leveraging data augmentation to enhance semantic representation and improve generalization. The adaptive keyword attention dynamically focuses on key industry terms, while the contrastive learning stage creates robust text embeddings despite the limited data. Experimental results demonstrate that our model achieves a substantial 4.8% improvement in ROUGE-L scores compared to the strong baselines on a limited enterprise news dataset. This highlights the effectiveness of our approach in mitigating overfitting and generating high-quality, informative summaries in low-resource scenarios.