Context-Aware and User Intent-Aware Follow-Up Question Generation (CA-UIA-QG): Mimicking User Behavior in Multi-Turn Setting

This paper introduces a Context-Aware and User Intent-Aware follow-up Question Generation (CA-UIA-QG) method in multi-turn conversational settings. Our CA-UIA-QG model is designed to simultaneously consider the evolving context of a conversation and identify user intent. By integrating these aspects, it generates relevant follow-up questions, which can better mimic user behavior and align well with users’ conversational goals. When assessed using public Shopping datasets on Fashion domain, our approach demonstrates significant enhancements over CA-QG baseline models. Specifically, it achieves an improvement of up to 3% in BLEU, 7% in METEOR, and 8% in ROUGE-Lsum. Additionally, our findings show the efficacy of fine-tuning in enhancing the model’s capacity to better mimic user behavior, CoT prompting with fine-tuned model yields superior performance compared to the ensemble method. Furthermore, we investigate the impact of model size, model type, and intent granularity, highlighting their impact to overall model performance. The importance of our work lies in its effectiveness to improve follow-up question generation from the user’s perspective and application in developing user-centric conversational AI systems.