Effects of Replacing Telephone with Web, Mail, and Mixed-Mode Data Collection in an Establishment Follow-Up Survey

Due to rising data collection costs, there is growing pressure to move away from traditional interviewer-administered mode designs in favor of fully self-administered mode designs in ongoing panel surveys, including large-scale establishment panels. However, the consequences of moving to a fully self-administered mode design on follow-up and cumulative participation in ongoing establishment panel surveys are largely unknown. To address this research gap, we report the results of a follow-up mode design experiment conducted in the second wave of the 2020 IAB-Job Vacancy Survey, an ongoing panel study in Germany. The experiment builds on a previously reported mode design experiment conducted in the first wave survey, where establishments were randomized to four self-administered mode designs (concurrent mail-web mixed-mode, sequential web-to-mail mixed-mode, single-mode web, and single-mode mail). In the second wave (i.e., follow-up) survey, reported here, respondents from the first wave were further randomly allocated to 1) a continuation of the same self-administered mode design from the first wave, or 2) a single-mode telephone design. The results show that the continuation of self-administration leads to higher response rates (both follow-up and cumulative) for the single-mode mail and concurrent mail-web mixedmode designs and comparable response rates for the single-mode web and sequential web-to-mail mixed-mode designs, compared to the telephone follow-up design. Using extensive administrative data, we do not find evidence that forgoing telephone follow-ups adversely affects nonresponse bias or subgroup participation compared to continuing with self-administration in the follow-up wave. Potential cost savings (of up to 67%) were evidenced when replacing the telephone mode with a self-administered followup mode design. In-depth qualitative interviews revealed that establishments prefer a constant mode design across waves due to the familiarity and routine of the response process.