Comparing the Convenience, Data Quality, Generalizability, and Outcomes of Student‐ and MTurk‐Generated Data in an Experimental Vignette Rape Perception Study

Vignette experiments assessing rape perceptions commonly use samples drawn from convenient sources, like university students or online crowdsourcing platforms like Amazon's Mechanical Turk. In the current study we compared the ease of data collection, cost, data quality, demographic characteristics, and experiment conclusions across these two sample sources, which were collected in a vignette experiment assessing mock jurors' perceptions of a hypothetical sexual assault. Results showed it was faster but more expensive to collect MTurk responses compared to student responses. Samples varied across several data quality measures, with students passing more manipulation checks and returning proportionately more usable cases than MTurkers. We also found considerable demographic differences between the samples, as well as rape myth acceptance (RMA) and victim blaming attitudes. Findings indicate that experiments' results and implications depend on sample source; pinpointing the factors that consistently influence rape perceptions, including RMA, will require replicating studies using diverse sample sources.