Validating a Japanese aviation English listening test: effects of pilot training experience and general English listening proficiency

This study presents an argument-based evaluation of the Aviation English Listening Proficiency Test (AELPT), developed by Civil Aviation College. Using an interpretation/use argument, it examined whether the available dataset supports three inferences for the intended interpretation of AELPT scores: evaluation, generalization across test forms, and extrapolation to theoretically relevant external variables. Data from 582 participants were analyzed using two-way analysis of variance to estimate the main and interaction effects of Pilot Training Experience (PTE) and general English listening proficiency (LP; TOEIC® IP Listening score). Both PTE and LP significantly affect test scores without any significant interaction between the two factors. Combined, the available evidence provides preliminary support for evaluation, generalization, and extrapolation inferences, indicating that PTE and LP are independently associated with AELPT scores. Future research can examine explanation and utilization inferences. The potential pedagogical implications of this study are particularly pertinent in English as a foreign language contexts (e.g., Japan), where the development and implementation of valid, operationally grounded testing systems remain challenging. Because the data were obtained from a single institution, generalizations should be made with caution.