A Multi-Scale Semantic Textural Similarity Model Based on Discrete Fourier Transform

The Semantic Textual Similarity (STS) task is a foundational problem in the field of natural language processing, with wide applications in search engines, recommendation systems, and retrieval-augmented generation for large language models. Although pre-trained language models based on the Transformer architecture have made significant progress in this task, they still face limitations such as parameter redundancy, difficulty in distinguishing subtle semantic differences, and inefficiencies in adapting large-scale models. This paper proposes MultiScaleBert, a multi-scale semantic similarity model based on Discrete Fourier Transform (DFT). By introducing a frequency-domain analysis mechanism, the model treats sentence representations as discrete signals and decomposes them into multi-scale semantic features, enabling the modeling of semantic information at different linguistic scales such as words, phrases, sentences, paragraphs, and articles, then MultiScaleBert reconstructs a fused global vector representation as the final output. Experimental results on multiple English STS datasets demonstrate that MultiScaleBert significantly improves the performance while maintaining a stable parameter scale, outperforming BERT across all five datasets and surpassing mainstream models of comparable scale, such as GTE and BGE, in overall performance. This validates the effectiveness and generality of the multiscale modeling approach in semantic textural similarity task.