Multi-Domain Dialogue State Tracking With Large Language Model Rationale and Disentangled Domain-Slot Attention

Dialogue state tracking (DST) is a core component of task-oriented dialogue systems. A significant challenge in this task is multi-domain DST, which involves considering dialogue states across multiple domains. Recent advancements have addressed this challenge by exploring various approaches to model the correlations among different domains with domain-slot-specific representations derived from dialogue context and aggregating domain-slot queries using sorts of attention mechanisms. However, existing models still exhibit deficiencies in handling these correlations, either by overlooking or overestimating them. In this paper, we propose a multi-domain DST framework with large language model (LLM) rationale and Disentangled Domain-Slot Attention to address this challenge. Specifically, we introduce a multi-domain aware instruction prompt to guide the LLM to generate the corresponding rationale to dialogue history, realizing these correlations along with the robustness against sorts of variations in spoken conversation. Additionally, we present a novel mechanism, termed Disentangled Domain-Slot Attention. This mechanism enables a dynamic, flexible, and context-dependent manner to extract domain-slot-specific information by disentangling domain-slot queries within the attention mechanism. Through extensive experiments on the MultiWOZ 2.0 and MultiWOZ 2.4 datasets, the results present that the proposed approaches improve the performance of multi-domain DST. In addition, we conducted empirical analyses to comprehensively understand the effectiveness of our proposed approaches.