Enhancing Chinese Dialogue Generation with Word–Phrase Fusion Embedding and Sparse SoftMax Optimization

Chinese dialogue generation faces multiple challenges, such as semantic understanding, information matching, and response fluency. Generative dialogue systems for Chinese conversation are somehow difficult to construct because of the flexible word order, the great impact of word replacement on semantics, and the complex implicit context. Existing methods still have limitations in addressing these issues. To tackle these problems, this paper proposes an improved Chinese dialogue generation model based on transformer architecture. The model uses a multi-layer transformer decoder as the backbone and introduces two key techniques, namely incorporating pre-trained language model word embeddings and optimizing the sparse Softmax loss function. For word-embedding fusion, we concatenate the word vectors from the pre-trained model with character-based embeddings to enhance the semantic information of word representations. The sparse Softmax optimization effectively mitigates the overfitting issue by introducing a sparsity regularization term. Experimental results on the Chinese short text conversation (STC) dataset demonstrate that our proposed model significantly outperforms the baseline models on automatic evaluation metrics, such as BLEU and Distinct, with an average improvement of 3.5 percentage points. Human evaluations also validate the superiority of our model in generating fluent and relevant responses. This work provides new insights and solutions for building more intelligent and human-like Chinese dialogue systems.