What Makes a Killer Question Killer? A Text Mining Analysis of High-difficulty Questions in the Korean CSAT English Section

This study investigated the thematic and lexical characteristics of high-difficulty English reading items—commonly referred to as killer questions—in the Korean College Scholastic Ability Test (CSAT) between 2018 and 2025. Using text mining methods, including Latent Dirichlet Allocation (LDA) and CEFR-based lexical profiling, the analysis reveals that these items disproportionately addressed abstract, academic topics such as cultural heritage, moral dilemmas, and cognitive science. Lexical analysis further indicated that over 30% of the words used are at the C1 level or above, reflecting substantial vocabulary demands. These findings point to a misalignment with the national curriculum, which is designed to target B1–B2 proficiency with more familiar and accessible topics. The combined challenge of abstract themes and advanced vocabulary raises concerns about the construct validity and fairness of these items, suggesting that they may measure background knowledge or test-taking strategies rather than genuine English reading proficiency. The study concludes by calling for evidence-based item development and closer alignment with curricular goals to ensure valid and equitable high-stakes language assessment.