A Domain-Finetuned Semantic Matching Framework Based on Dynamic Masking and Contrastive Learning for Specialized Text Retrieval

Semantic matching is essential for understanding natural language, but traditional models like BERT face challenges with random masking strategies, limiting their ability to capture key information. Additionally, BERT’s sentence vectors may “collapse,” making it difficult to distinguish between different sentences. This paper introduces a domain-finetuned semantic matching framework that uses dynamic masking and contrastive learning techniques to address these issues. The dynamic masking strategy enhances the model’s ability to retain critical information, while contrastive learning improves sentence vector representations using a small amount of unlabeled text. This approach helps the model better align with the needs of various downstream tasks. Experimental results show that after private domain training, the model improves semantic similarity between entities by 16.9%, outperforming existing models. It also demonstrates an 8.0% average improvement in semantic matching for diverse text. Performance metrics such as A@1, A@3, and A@5 are at least 26.1% higher than those of competing models. For newly added entities, the model achieves a 44.3% average improvement, consistently surpassing other models by at least 30%. These results collectively validate the effectiveness and superiority of the proposed framework in domain-specific semantic matching tasks.