TEMDG: A Multi-Round Dialogue Generation Model Incorporating Topic and Historical Information

This paper proposes a Topic-Enhanced Multi-Turn Dialogue Generation Model (TEMDG), which intends to solve the problem of insufficient context as well as topic consistency in the current dialogue models. Firstly, this paper adopts the bi-term topic model (BTM) to extract implicit topics from the corpus, and secondly, in the encoding of historical information, hierarchical recursive coding is used and the computation of word-level attention is introduced into the utterance encoding process. This solves the problem of losing historical information over long distances in hierarchical cyclic coding. The extracted topics are then subjected to the computation of correlation with the history-encoded sentence vectors, aiming to ensure that the resulting topic representation can contain key information related to the dialogue history. Finally, through the fusion attention mechanism, the historical and topic information is fused and decoded in order to generate the corresponding responses. The experimental results show that the TEMDG model outperforms the other baseline models on the DailyDialog dialogue dataset, especially on the diversity metrics showing a significant improvement of 16.48% compared to the best performing model in the baseline. This indicates that the TEMDG model is able to integrate more information, including topic information and historical dialogue information when generating replies. Sample analyses revealed that the model-generated responses better incorporated topic information, improving the accuracy and relevance of the responses. And in the manual evaluation, the percentage of our model reached 22.20% under the scoring criterion indicating consistency in topic relevance, which is also significantly better than the other models. Based on the results of both automatic and manual evaluation, the TEMDG model is able to consider the topic information and historical dialogue information better when generating replies and generates more natural, smooth, and consistent replies, which effectively solves the problem of lack of consistency of context and topic in the current dialogue model, and significantly improves the quality and consistency of the dialogue generation.