Unanticipated baseline differences: evidence of expectancy bias in an Adaptive Mobile Learning intervention in Sub-Saharan Africa

This study provides empirical evidence of expectancy bias emerging before an educational technology intervention in a Sub-Saharan African university context. Within a larger quasi-experimental evaluation of an Adaptive Mobile Learning System (AMLS), a pre-intervention baseline assessment (Survey 2) was conducted to establish group equivalence. However, following procedural disclosure rather than intentional priming, the experimental group, informed in advance of their forthcoming AMLS use in Phase 3, recorded significantly higher baseline scores than the control group across cognitive learning outcomes, engagement, and adoption perceptions, despite both groups using identical non-digital study materials. Independent-samples t-tests (α = .001) and Cohen’s d indicated consistent small-to-moderate effects (d ≈ 0.35–0.45) across domains for N = 476 (control) and N = 485 (experimental). This pattern is interpreted primarily as expectancy bias, supported by complementary lenses from the Hawthorne effect (enhanced performance due to awareness), placebo-like expectancy in learning (performance shaped by anticipated novelty), and demand characteristics (responses aligned with perceived expectations). The findings highlight methodological risks for educational technology research, particularly in low-resource settings where anticipatory expectations may distort baseline assessments. Practical recommendations are offered to strengthen experimental rigor and improve interpretation of intervention effects.