Preference inversion in discrete choice experiments: A novel test of the emoji scale

Preference inversion - where preference estimates do not align with the hierarchical order of attribute levels-remains a concern in health valuation studies. To address this, we developed a novel visual aid, the emoji scale, comprising five Apple Unicode emoji characters to represent five levels of a health-related quality-of-life measure. Using a case study of the Recovering Quality of Life - Utility index instrument, we explored whether the emoji scale could reduce preference inversion and improve respondent experience in discrete choice experiments (DCEs). We conducted a between-respondent comparison involving 2024 Australian adults, randomly allocated to one of four arms:(1) standard text-only choice tasks, (2) emoji & text (combined), (3) emoji-only (no text), and (4) text with purple colour-coding. We estimated value sets derived from each format and assessed differences in perceived task difficulty, engagement, and preference inversion using t-tests and model comparisons. Results showed that utility decrements from the emoji & text were closely aligned with those from the standard text-only format. Importantly, the emoji & text reduced preference inversion and was associated with lower reported task difficulty and higher engagement compared to other formats. These findings suggest that pairing emojis with text may enhance respondent understanding and data quality in health valuation tasks, without compromising the validity of the resulting value sets. While further work is necessary to fully understand the legitimacy of the emoji scale as an effective visual aid for conveying attribute level ordering, the emoji & text format offers a promising approach to improving DCEs.