Weak-to-strong boundary learning: leveraging pre-trained models to enhance large language models’ entity boundary recognition

Entity boundary recognition is a critical component for large language models in natural language processing tasks, playing a vital role in accurately understanding textual semantic structures and extracting key information. This paper addresses the challenges of inaccurate boundary localization in large language models and their inability to fully leverage the specialized advantages of pre-trained models in boundary recognition, proposing an optimization method for entity recognition in large language models based on weak-to-strong boundary learning. We construct a weak-to-strong learning framework based on three stages: “pre-trained model boundary recognition training, boundary alignment signal extraction, and large model boundary learning optimization,” which directly transfers the boundary recognition capabilities and improvement information from pre-trained models to large language models; We design a multi-dimensional fusion boundary improvement metric function that comprehensively quantifies the enhancement of boundary recognition capabilities in pre-trained models from three dimensions: boundary gain, confidence change, and dynamic weight adjustment, ensuring effective transmission of alignment signals through normalization and type-sensitive weight allocation mechanisms; We propose a loss function optimization strategy with embedded boundary alignment signals, combining boundary improvement metrics with the generation probability distribution of large models to achieve precise boundary learning and capability transfer based on alignment signals. Experimental results demonstrate significant differences in the effectiveness of various alignment signal design strategies, with the complete diff_score function achieving an average improvement of 1.8% over single-component approaches, and the dynamic weight mechanism showing the most significant improvement in boundary recognition for organization entities; The adjustment of hyperparameter $$\gamma $$ significantly impacts model performance, achieving optimal balance at $$\gamma $$=0.6 with a boundary F1 score of 88.5% and a language modeling perplexity of only 3.42. In terms of boundary recognition performance, LLaMA3.1-8B, Qwen3-8B, and ChatGLM4-9B models based on the W2S-BL method achieve overall boundary F1 scores of 88.5%, 89.6%, and 87.3% respectively on the CoNLL-2003 dataset, representing improvements of 5.5%, 4.8%, and 4.7% respectively compared to direct training methods; Performance improvements are significant across different entity types, with organization entity boundary recognition showing the largest improvement of up to 6.4%. The method proposed in this paper provides a new technical pathway for optimizing the performance of large language models in entity recognition tasks, offering broad application prospects in information extraction, knowledge graph construction, and intelligent question answering. Code and implementation details are available at https://huggingface.co/wmsr22/W2S/tree/main