Planner Policies are Bias Learners: A Comprehensive Analysis of Preference Bias in LLM-based Dialogue Planner Policies

Large language models have facilitated the development of sophisticated dialogue planner policies. However, their reliance on LLM-based simulation and feedback for policy optimization may introduce preference bias. This paper presents the first comprehensive analysis of preference bias in LLM-based dialogue planners. It systematically evaluates four state-of-the-art planning methods across three dialogue domains, utilizing multiple LLM families at varying scales. Our investigation reveals that all tested planners exhibit significant preference bias, favoring narrow strategy sets over maintaining balanced distributions. Notably, larger models paradoxically amplify bias. User simulation emerges as the primary bias driver, while diverse persona simulation proves ineffective as a mitigation strategy. Most concerning, preference bias leads planners toward ethically problematic strategies that achieve short-term success while compromising real-world effectiveness and ethical standards. These findings establish fundamental challenges for responsible deployment of LLM-based dialogue systems and provide crucial insights for developing more reliable and ethically aligned planning approaches.