A comparative analysis of differential item functioning in ePIRLS 2021: traditional and intersectional approaches

There is a large and growing body of literature describing the role of differential item functioning (DIF) in test development and validity. While traditional DIF analyses typically examine demographic variables independently, the intersectional approach aims to provide a more comprehensive investigation of test bias by accounting for simultaneous impact among these variables. This is particularly important in large-scale computer-based assessments, where the factors influencing student performance have become increasingly complex. Rendering traditional DIF analyses potentially inadequate for fully detecting item bias. This study explores whether the computer-based reading literacy assessment administered across countries in the digital-based PIRLS 2021 cycle shows DIF with respect to gender, home socioeconomic status, and digital self-efficacy, employing both traditional and an intersectional DIF framework. The analysis includes 7134 students from 17 countries that participated in the ePIRLS assessment. DIF was examined using a comparative analysis of the Mantel-Haenszel, SIBTEST, and Multiple Indicators Multiple Causes methods. The findings revealed that while the traditional approach identified only negligible DIF, the intersectional approach revealed items with moderate DIF, demonstrating its enhanced sensitivity in detecting more substantial sources of bias. The results also indicated that the intersectional approach across all three methods flagged a greater number of items as potentially biased compared to the traditional approach. Thus, this study contributes to the literature by providing an in-depth examination of DIF and highlighting the relevance and applicability of intersectional DIF analyses in large-scale assessments.