Argument-based validation of Chulalongkorn University Language Institute (CULI) test: a Rasch-based evidence investigation

Chulalongkorn University Language Institute (CULI) test was developed as a local standardised test of English for professional and international communication. To ensure that the CULI test fulfils its intended purposes, this study employed Kane’s argument-based validation and Rasch measurement approaches to construct the validity argument for the CULI test. This study analysed score data from a single test administration involving 237 test-takers and used a dichotomous Rasch model for the data analysis in Winsteps. The Rasch results indicated appropriate psychometric properties of the CULI test. Overall, the Rasch-based evidence reasonably contributes to the validity argument for the CULI test. Specifically, it partly supports the plausibility of the claims that CULI test tasks and contents represent those in the intended target language use (TLU) domain, and CULI test scores accurately summarise test-takers’ performance and reflect test-takers’ performance consistency, the intended construct, and language performance levels in the TLU domain. Although the CULI test showed sound psychometric functioning, the Rasch results indicated the need for further investigation into certain correct and incorrect choices and the difficulty levels of writing items, which could provide valuable insights for optimising test quality. This study offers implications for future research on argument-based and Rasch-based validation.