A Reinforcement Learning-Based Retraining-Free Pruning for Encoder-Based Language Models

Natural Language Processing (NLP) has achieved significant success in complex tasks across various domains, yet it also brings high computational costs and inference delays. Pruning, as a model optimization technique, can effectively reduce model complexity and enhance its generalization capability and efficiency. However, current encoder-based language model pruning algorithms often lack robust dynamic adaptability and tend to focus only on short-term optimal solutions, without fully considering the interactions between different solutions. This limits their ability to find global optima, thereby potentially impacting overall model performance. To address these challenges, we propose a structured pruning algorithm based on reinforcement learning, named RLM (Reinforcement Learning Masking), which includes QLOM (Q-Learning Optimization Mask) and QRMT (Quasi-Minimal Residual Mask Tuning) components. This algorithm aims to rapidly and effectively find global optima without the need for retraining. We evaluated this method using BERT<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BASE</inf> and DistilBERT models on the GLUE and SQuAD benchmarks. Experimental results show that RLM significantly enhances model accuracy in the SQuAD benchmark. Under a 60% FLOPs constraint, RLM achieves a 8.45% increase in F1 score compared to existing retraining-free pruning algorithms, demonstrating its effectiveness in improving performance while managing computational resources efficiently.