Replicating Semantic Priming Online: A Historical Comparison of In-Lab and MTurk Data Quality

Semantic priming in lexical decision tasks has long provided evidence for automatic spreading activation, yet questions remain about how robust these effects are across different research contexts. The present study compared priming outcomes for undergraduates tested in the laboratory and participants recruited via Amazon Mechanical Turk in the early 2010s, using a Qualtrics-based lexical decision task. Reaction times (RTs) were trimmed at ±3 SD within participants, and accuracy was calculated as the proportion correct. A mixed-design ANOVA revealed the expected main effect of prime type, with both groups showing the classic hierarchy of associated < unassociated < nonword stimuli. Within each setting, planned comparisons confirmed a full three-level gradient, underscoring the robustness of semantic priming even when measured online with tools not optimized for RT accuracy. A small but reliable interaction indicated that the overall priming span (nonword − associated difference) was larger for lab participants (M = 0.25 s) than online participants (M = 0.17 s), Hedges’ g = 0.50. Contrary to predictions, accuracy was significantly higher for online participants (M = .93) than lab participants (M = .89), g = −0.76. Perceptions of being observed did not differ between settings. Although the study was not preregistered, all data, materials, and analysis scripts are openly shared. Taken together, these findings demonstrate that semantic priming replicates robustly across both laboratory and early online contexts, highlight a modest laboratory advantage in effect magnitude, and illustrate how basic cognitive phenomena can withstand technically imperfect measurement conditions.