Measuring grammatical understanding

Abstract In the realm of first language (L1) grammar education, research on instruments assessing explicit grammatical understanding is limited. This article discusses an Item Response Theory (IRT) analysis on the Test for Grammatical Understanding (TGU), a multiple-choice tool designed for measuring such grammatical understanding, in this case pertaining to the concept of valency and related concepts (e.g., subject, objects). The results draw from 189 pre-university students (aged 14), who completed different versions of the TGU at multiple time points. Using IRT models for categorical data, we assessed item difficulty, discrimination, and response category functioning. Results highlight the TGU’s usability, version interchangeability, and its potential as a tool for monitoring progress in grammatical understanding. Additionally, differential item functioning (DIF) analyses indicated that most test items performed consistently across versions, though a few showed discrepancies requiring further examination. The discussion addresses item success variations, proposing items that might be omitted under time constraints, and examines the value of a ‘partial understanding’ answer category alongside categories reflecting ‘full’ and ‘no understanding.’ Future study recommendations include expanding TGUs to different linguistic concepts beyond valency and exploring its correlation with open reasoning tasks to further validate its effectiveness in assessing explicit grammatical understanding.