The reduction of heaping in numeric open-ended questions through immediate interactive feedback: experiments in a web survey

In open-ended numeric survey questions, certain numbers, typically round numbers, are disproportionately mentioned, creating unsubstantial “heaps” in the response distribution. These distortions often arise from satisficing behavior or imprecise memory, making it difficult to correct the resulting measurement error through post-hoc weighting. With the intention to reduce the incidence of heaping during data collection, we conducted an experimental study embedded in a web survey of the general internet population in Germany. Respondents answered two list-style numeric open-ended questions and were randomly assigned to either an experimental group, where round-number responses triggered immediate interactive feedback, or a control group getting no feedback. Results show that the feedback in the form of instructing and appreciating text-bubbles significantly reduced both the prevalence of round answers and the degree of roundness of the responses. This indicates that interactive feedback can effectively mitigate heaping in numeric open-ended questions.