Impact of envelope teaser on mail and web research survey response rates and representation

A high response rate is an often-used measure of survey quality. Research is mixed on the effectiveness of a “teaser message” to increase response rate, with at least one study showing a significantly > 10% higher response rate with a teaser compared to control and others showing no significant differences. As part of a larger study of care coordination, 6813 individuals were mailed a push-to-web survey invitation letter, half randomized to receive an envelope with the teaser “We need your help!” stamped on the outside of the envelope in red ink, and half without. A second mailing with a paper survey was sent to initial non-responders with the teaser condition carried through. Survey response rates were calculated using AAPOR Response Rate 6 and differences between teaser and control respondents were compared using Chi-square tests. Impact of teaser on response rate within patient subgroups was calculated using the same analytic approach applied to the overall sample. In total, 2795 individuals completed the survey (41% response rate). After the initial mailing containing the push-to-web invitation, the response rate was 13% for both the teaser and control groups (p = 0.73). After the second mailing, there were again no significant differences in response rate (teaser 34%, control 36%, p = 0.07). The response rate was lower among white (4% lower, p = 0.01), female (3% lower, p = 0.02), urban individuals (2% lower, p = 0.04), and those over age 65 (4% lower, p = 0.02) in the teaser group than control group. While the existing literature is inconclusive and nuanced, our findings align with standard survey methods which recommend against using colored and catchy text on survey envelopes. Based on these findings, we do not recommend the use of an envelope teaser message. All methods were reviewed and approved, with waiver of written consent, by the HealthPartners Institutional Review Board (#A19-119, 07/20/2022).