Study and instrument quality in perception-based L2 pronunciation research

Abstract This methodological synthesis surveys study and instrument quality in L2 pronunciation research by scrutinizing methodological practices in designing and employing scales and rubrics that measure accentedness, comprehensibility, and intelligibility. A comprehensive coding scheme was developed, and searches were conducted in several databases. A total of 380 articles (409 samples) that employed 576 target instruments and appeared in peer-reviewed journals from 1977 to 2023 were synthesized. Results demonstrated, among other findings, strengths in reporting several listener and speaker characteristics. Areas in need of improvement include (a) more thorough evaluation and reporting of interrater reliability and instrument validity and (b) greater adherence to methodological transparency and open science practices. We conclude by discussing the implications of these findings for researchers and researcher trainers; by raising awareness of methodological and ethical challenges in psychometric research on L2 speech perception; and by providing recommendations for advancing the quality of instruments in this domain.