Long Dialogue Segmentation Based on Memory Similarity Algorithm

In the digital era, with the widespread use of the internet and accelerated digitalization, the speed of information generation and dissemination has significantly increased, leading to the challenge of information overload with massive data and lengthy texts. This paper proposes a BiLSTM-based similarity calculation model designed to achieve automatic segmentation of long dialogues while maintaining semantic similarity in the segmented text. We conducted a series of experiments on the DialSeg711 and Doc2Dial datasets and compared our model with various state-of-the-art dialogue segmentation models, including BayesSeg, GraphSeg, TextTiling, and its improved versions: TextTiling + Glove and TextTiling + Embedding. Using Pk and WD as primary evaluation metrics, we systematically assessed the performance of the proposed model in terms of topic consistency and segmentation accuracy. The experimental results demonstrate that the proposed BiLSTM-COS method shows significant advantages in the task of topic segmentation in long dialogues, effectively capturing contextual information within conversations and substantially improving semantic consistency and segmentation accuracy.