The Partner Modelling Questionnaire: A Validated Self-Report Measure of Perceptions toward Machines as Dialogue Partners

Recent work has looked to understand user perceptions of speech agent capabilities as dialogue partners (termed partner models), and how this affects user interaction. Yet, partner model effects are currently inferred from language production as no metrics are available to quantify these subjective perceptions more directly. Through three phases of work, we develop and validate the Partner Modelling Questionnaire (PMQ): an 18-item self-report semantic differential scale designed to reliably measure people’s partner models of non-embodied speech interfaces. Through confirmatory factor analysis, we confirm that the PMQ scale consists of three factors: communicative competence and dependability, human-likeness in communication and communicative flexibility. Our studies show that the measure consistently demonstrates good internal reliability, strong test-retest reliability over 4- and 12-week intervals, and predictable convergent/divergent validity. Based on our findings, we discuss the multidimensional nature of partner models, while identifying key future research avenues that the development of the PMQ facilitates. Notably, this includes the need to identify the activation, sensitivity, and dynamism of partner models in speech interface interaction.