Psychometric Foundation for Digital Assessment

<p><span style="color: rgb(59, 56, 56);">This study developed and validated a psychometrically robust formative assessment instrument to measure elementary students’ conceptual understanding of material properties in Natural and Social Sciences (NSS), establishing essential foundations for technology-enhanced assessment systems. Employing Tessmer’s development model integrated with Rasch analysis, the research implemented a multistage validation process involving expert reviews (Aiken’s </span><em style="color: rgb(59, 56, 56);">V</em><span style="color: rgb(59, 56, 56);"> = 0.931), cognitive testing with diverse learners (</span><em style="color: rgb(59, 56, 56);">n</em><span style="color: rgb(59, 56, 56);"> = 3), small-group trials (</span><em style="color: rgb(59, 56, 56);">n</em><span style="color: rgb(59, 56, 56);"> = 10), and large-scale field testing (</span><em style="color: rgb(59, 56, 56);">n</em><span style="color: rgb(59, 56, 56);"> = 171 students from eight Indonesian schools). Quantitative psychometric analysis demonstrated high reliability (person reliability = 0.807) and appropriate item difficulty distribution (−2.12 to +1.78 logits), while Mantel-–Haenszel Differential Item Functioning (DIF) analysis revealed significant gender bias in Item 20 (χ² = 6.53, </span><em style="color: rgb(59, 56, 56);">p</em><span style="color: rgb(59, 56, 56);"> = 0.011; ΔMH = −2.56), where female students exhibited 47% lower success probability, despite equivalent ability levels. The instrument meets rigorous psychometric standards for 97% of items, though Item 20 requires contextual revision to eliminate domestic stereotypes compromising fairness. Critically, this study provides: (1) calibrated item parameters ready for integration into adaptive learning algorithms, and (2) an evidence-based bias screening protocol for EdTech assessment design. These contributions advance gender-responsive science assessment practices while enabling future development of fair, technology-driven evaluation tools in Science, Technology, Engineering, and Mathematics</span><strong style="color: rgb(59, 56, 56);"> </strong><span style="color: rgb(59, 56, 56);">(STEM) education.</span></p>