Teacher-Guided History-Aware Negative Sampling for Conversational Retrieval in Domain-Specific Chatbot

Delivering accurate and contextually relevant information in domain-specific conversational agents presents significant challenges, particularly in multi-turn dialogues, where the resolution of query ambiguity and the effective utilization of conversation history remain critical issues. Existing research has primarily concentrated on English datasets, with limited efforts to integrate the resolution of conversational query ambiguity into the model training process. This study seeks to address this gap by (1) establishing a baseline for conversational retrieval models using a domain-specific Korean conversation dataset, in comparison to an English benchmark dataset, and (2) proposing a Teacher-Guided History-Aware Negative Sampling method to enhance retrieval performance. Our two-phase approach begins with the training of a teacher model using baseline negative sampling methods. Subsequently, we leverage this model to identify and incorporate distracting documents from the conversation history as negative samples, thereby training the model to concentrate on the user's current intent while disregarding irrelevant context. Experimental results demonstrate that our method surpasses baseline models, achieving enhanced performance across key retrieval metrics such as Recall and Mean Reciprocal Rank (MRR). These findings contribute to the advancement of retrieval methods in specialized domains by addressing the unique challenges posed by the Korean language and the complexity of dialogue, while also providing valuable insights through comparisons with English benchmark datasets.