Guidance or Misdirection? Unpacking the Role of Feedback in Health Preference Assessments

This study investigated the impact of providing feedback to respondents on a dominance-structured choice task on subsequent choice behavior in a discrete choice experiment (DCE). The DCE was conducted among 626 patients with heart failure. Respondents were given a dominance-structured choice task in which two devices (Device A and Device B) offered no benefits but carried risks compared to a "No Device" option. Among those who selected a device option (N = 340), half received feedback and an opportunity to revise their choice, while the other half did not. The effect of feedback on preference for the "No Device" option and choice consistency was examined using multinomial, heteroscedastic multinomial logit, and heteroscedastic latent-class logit models. Among those who received feedback (N = 170), 71% continued to choose the device options. Feedback recipients were more likely to choose the "No Device" option in subsequent questions (p < 0.01). Feedback led to a 25% reduction in choice consistency (p < 0.01) and an increased likelihood of choosing the "No Device" option. Impact on consistency varied across latent classes: feedback decreased consistency in the risk-sensitive class but increased consistency in the anti-device class, highlighting potential unintended consequences. Further research is needed to understand its effects in different contexts and samples.