Seeing emerging comprehenders: Refining reading comprehension assessment in international contexts

There is little consensus on the optimal approach to measuring reading comprehension in low- and middle-income countries (LMIC), and significant scholarly debate surrounds the approach used by the Early Grade Reading Assessment (EGRA). Concerns focus on the EGRA task’s conflation of reading speed with comprehension and the high rates of zero scores it produces. This study investigates three reading comprehension measures among 3rd and 4th grade students in Mali and Senegal, across their first and second languages. The approaches include the standard EGRA task (a timed oral reading fluency plus comprehension measure), a modified version allowing students to look back at the text to answer questions, and a new, untimed picture-matching task. We employ Item Response Theory analysis to assess item difficulty and discrimination, and quantile regression analysis to examine relative sensitivity to students’ literacy sub-skills across tasks and languages. Results show that the picture-matching task more effectively measures comprehension among students at lower ability levels and is highly sensitive to slow decoders, while the standard EGRA task is more sensitive to strong decoders but fails to capture slow readers' comprehension skills. We conclude that combining the EGRA and picture-matching tasks provides a broader and more accurate measurement of students' reading comprehension abilities, helping to correct key limitations of the standard EGRA approach. These findings have significant implications for policy and practice and better understanding LMIC students’ reading abilities, progress, and the efficacy of literacy interventions. • EGRA fails to detect the reading comprehension (RC) skills of slow decoders. • High ratios of LMIC students show comprehension on a scaffolded, untimed assessment. • Tasks have variable sensitivity to students RC skills with high versus low decoding. • EGRA may underestimate programs’ reading comprehension effects on slow decoders. • EGRA’s comprehension test may overstate reading comprehension failure in LMICs.