Validating a diagnostic reading test for junior high school EFL learners in Indonesia's English massive program using QUEST

Background: Non-formal English education programs often lack rigorous assessment tools, resulting in challenges in evaluating student performance and guiding instructional improvements. One such program, the English Massive Program (EMAS) in Kediri, East Java, Indonesia, serves as a community-driven initiative focused on enhancing English proficiency. However, the quality of its assessments, especially in reading comprehension, remains a critical concern. Aims: This study aims to investigate how item analysis using the QUEST application can enhance the quality of diagnostic assessment and instructional strategies in non-formal English education, specifically within the EMAS program. The focus is on analysing reading comprehension tests to identify weaknesses and propose improvements in test construction. Methods: This exploratory study analysed 30 multiple-choice reading comprehension items completed by 26 junior high school students participating in the EMAS program. The QUEST application was employed to assess item difficulty, discrimination, and distractor efficiency. Results: The results showed that while most discrimination indices were within acceptable ranges, many items, especially the distractors, were too simple and ineffective. This resulted in insufficient and unbalanced discrimination values, indicating that the test items did not optimally differentiate among varying student ability levels. Implications: The study underscores the importance of integrating psychometric-based diagnostic tools in community education settings. It demonstrates how such analysis can empower educators with practical insights to improve test design, thereby enhancing assessment quality and pedagogy. The research calls for more advanced diagnostic assessment methods to support literacy and instructional planning in low-resource, non-formal educational environments.