The Effect of Communication Emails on Web Survey Response Rate, Representativeness, and Response Bias: Results from a Factorial Randomized Control Trial in a College Student Population

Web surveys are popular in social sciences for reaching a large audience at a low cost and in a short period. However, response rates and nonresponse bias are still issues of concern. Using a factorial randomized control trial design, this study explores whether different communication emails affect response rates, representativeness, and response bias in a web survey of doctoral students enrolled in a U.S. university. A total of 5,539 doctoral students were stratified and randomly assigned to one of six groups. Results showed that none of the interventions increased the overall response rate. Moreover, interventions affect the representativeness of the sample and are associated with response bias.