A construct validation study of time-sensitive word-knowledge measures

Abstract Researchers increasingly use response time tasks, along with untimed, accuracy-based tasks, to measure lexical knowledge. While these tasks are set up differently and are inherently time-sensitive, there is currently no empirical evidence on whether they also tap into qualitatively different dimensions of vocabulary knowledge. In this article, we report a validation study involving five timed and untimed measures for assessing 40, 2K–5K frequency English words. One hundred and forty-five learners took (1) an untimed meaning recognition test, (2) an untimed form recall test, (3) a Yes–No response time (RT) test (affording both accuracy and RT measures), and (4) a masked repetition priming task. Confirmatory factor analysis suggests that these measures can be placed on one or two psychometric dimensions, whereby the one-factor solution (suggesting the tests measure one construct) is preferred for parsimony. However, the two-factor model had a marginally stronger predictive validity for explaining self-reported proficiency. Our results highlight the value of incorporating response time measures in vocabulary research although the jury is still out on the theorization of vocabulary knowledge as a one- or two-dimensional construct.