Fraud, Effort, and Data Quality Across Four Online Recruitment Modes: A Head-to-Head Protocol Comparison for a Minority Subpopulation Cohort Study

Comparisons of online recruitment sources are rarely conducted under a unified protocol; literature-based comparisons confound recruitment mode with variation in eligibility criteria, survey instruments, incentives, and data-quality controls. We compared four concurrently fielded recruitment modes (text-message invitations, targeted digital advertising, social media advertising, and a commercial nonprobability panel) using a common screener, instrument, incentive, and quality-flagging system to establish a large cohort of a minority subpopulation: young adults (aged 18-25) in California who use electronic nicotine delivery systems. Screener completion per invitation/impression ranged from 0.8% to 11.1% with 12,547 completed screeners. Two quality-flagging systems were applied with 806 retained respondents. Screener fraud flags were 4 to 9 times more frequent for social media advertising than for any other mode (p < .001). Across all recruitment modes except social media, endorsing eligibility items was associated with a higher likelihood of screener flags, with one exception (ps < .008), a pattern consistent with eligibility fraud. Baseline assessment insufficient effort responding (IER) flags were most frequent among panel respondents (p < .001) and least frequent among respondents recruited via text-message and targeted digital advertising, the two probabilistic sampling modes. Paradata-based and content-based screening were complementary rather than interchangeable data-quality tools with weak overlap (Jaccard index = .20); most respondents exceeding the baseline threshold were already excluded via screener flags, whereas most screener-flagged respondents produced internally consistent reports, indicating the screenings captured distinct validity threats. Recruitment mode systematically shaped both fraudulent responding and response effort. Social media advertising yielded the largest and most demographically diverse sample but also the highest screener fraud rates; the commercial panel showed little evidence of fraud but the highest IER. Multi-modal recruitment of minority subpopulations into large longitudinal studies is feasible and, with layered quality controls, can yield cohorts with acceptable fraud and IER profiles.