Examining the Factors Behind Experimental Group Superiority in Quasi-Experimental Research: A Mixed-Methods Analysis of MA Theses

When random assignment is not feasible in education research, quasi-experimental designs are used to examine the impact of instructional interventions on other variables. Notably, a recurring pattern in such studies is the outperformance of the experimental groups over the control groups, which raises questions about the validity and interpretation of these outcomes. This study investigated the factors that lead to the consistent outperformance of experimental groups over control groups in quasi-experimental research. By adopting a mixed-methods approach, the researchers analysed 100 MA theses of the Department of Curriculum and Instruction at Sultan Qaboos University in the Sultanate of Oman. This analysis was supported by a survey that was distributed to lecturers at four universities in the country to elicit their insights on issues that include selection bias, intervention effectiveness, implementation differences, and the Hawthorne effect, as potential contributors to this trend. Based on the survey responses of the lecturers, the primary contributing factor to the superior performance of experimental groups compared to control groups was the Hawthorne effect, followed by the effectiveness of the intervention and selection bias as secondary factors, accounting for the observed trend. The findings offer a comprehensive explanation of the underlying causes and present practical recommendations for improving the design and interpretation of quasi-experimental studies in education research.