Authenticity of Lesotho’s SACMEQ II-IV literacy and numeracy reports

Abstract The Southern and Eastern Africa Consortium for Monitoring Educational Quality, now named the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SEACMEQ), has conducted successive regional assessments of Grade 6 literacy and numeracy. This study examined a defined national reporting corpus: Lesotho’s SACMEQ II (2000), SACMEQ III (2007), and SACMEQ IV (2013) national reports. It did not evaluate SEACMEQ V, international reports as a complete series, or the psychometric validity of the underlying tests. Using an interpretivist qualitative case study, we combined documentary analysis with semi-structured interviews involving four purposively selected senior stakeholders. We developed a provisional Authenticity Evaluation Framework from four reporting functions identified in assessment, validation, evaluation, and evidence-use literature: methodological transparency, fidelity of representation, contextual interpretation, and policy accessibility and usefulness. Inductive content analysis, supported by ATLAS.ti 24, indicated that the national reports were regarded as methodologically credible and useful for monitoring broad performance trends. Their national interpretive value was nevertheless limited by uneven contextual explanation, incomplete discussion of the boundaries of the sampled population, restricted dissemination, and limited transparency about some reporting limitations. These findings do not challenge the technical validity of SACMEQ scores. They show that technically defensible results can require further national interpretation before they adequately support context-sensitive policy decisions. The framework is therefore presented as a provisional analytical heuristic, not a validated measurement model. Its transferability and construct validity require examination in larger, multi-country studies.