Transformer-based text summarization with dynamic hybrid meta-heuristic learning rate optimization (ICA, GWO, and PSO)

Transformer-based abstractive text summarization models are highly sensitive to hyper-parameter configurations, particularly the learning rate, which critically influences convergence dynamics and summary quality. Conventional tuning strategies—such as grid search, random search, and deterministic schedulers—often fail to adapt effectively to the highly non-convex optimization landscape of transformer training. This paper proposes a dynamic hybrid meta-heuristic optimization framework that integrates the Imperialist Competitive Algorithm (ICA), Grey Wolf Optimizer (GWO), and Particle Swarm Optimization (PSO) to adaptively tune the learning rate of a T5-base model. We introduce a phased hybrid strategy (ICA-GWO) that allocates 30% of iterations to global exploration and 70% to local exploitation. Unlike existing hybrid meta-heuristics, our approach uses temporally segregated, role-specialized hybridization where ICA operates exclusively during exploration and GWO exclusively during exploitation, with a theoretically motivated fixed-phase transition. To our knowledge, this is the first such application to transformer learning rate optimization for abstractive summarization. We evaluate our approach on CNN/DailyMail and XSum datasets. The optimized models achieve significant ROUGE-L improvements over baseline methods and demonstrate improved generalization under domain shift. Human evaluation and BERTScore validation confirm reduced factual errors. Our T5-base model achieves competitive performance compared to larger models such as T5-large, while maintaining lower computational cost. These results demonstrate that adaptive meta-heuristic optimization can rival model scaling, providing a practical pathway for high-quality summarization in resource-constrained environments.