Exploring the Validity of the Advanced Placement (AP) Japanese Computer-Simulated Interpersonal Speaking Task

This study examined the domain definition inference for the AP Japanese exam, which aims to measure language proficiency in situations reflecting real life. For the domain definition inference, this exam attempts to claim that (a) features of examinee performance that provide evidence of language proficiency have been identified, and (b) the assessment tasks are relevant to and representative of the real-life domain. Focusing on the computer-simulated conversation task of the AP Japanese exam, four assumptions underlying these claims were investigated: (1) the target domain is clearly identified; (2) critical language skills and knowledge needed in the target domain are identified; (3) assessment tasks that are representative of the target domain are identified; and (4) assessment tasks that require critical skills and are representative of the target domain are simulated. Backing for these assumptions was gathered through analyses of exam-related documents and test task characteristics, including context, input language, and scoring criteria. This study found reasonable backing for the first two assumptions and partial backing for the others. Therefore, this paper argues that the two claims that the exam attempts to make for the domain definition inference are not fully supported. It also discusses possible ways to improve the validity argument.