Automatic cross-lingual annotation of clause-based utterance fluency in L2 speech
Automatic annotation of temporal speech features can scale second language (L2) fluency research. However, existing approaches are validated mainly on English and vary in accuracy across measures. We present a cross-lingual pipeline that computes utterance fluency measures from audio and operationalizes the Vercellotti and Hall (2024) clause framework. We validate it on 40 L2 English and 40 L2 Japanese storytelling monologs. Evaluation follows a staged design: clause-boundary agreement, pause location classification, and concurrent validity with manual reference measures. Reference clause annotations combined independent double coding, large language model-assisted segmentation, coder review, and an audio check. In both languages, the clause boundaries placed by the pipeline agreed well with the manual reference. The pipeline’s classification of pauses as mid-clause or end-clause agreed closely with the reference in English and well in Japanese. All nine automatic measures correlated strongly with the manual measures in both languages. In a matched English comparison, the pipeline measured articulation rate and pause ratio with much smaller errors than existing acoustic and transcription-based tools. Among the clause-based measures, mid-clause pause ratio gained the most over the transcription-based tool. Speech rate, mean length of run, and overall pause ratio were the most robust measures. End-clause pause ratio was the weakest. These findings are limited to monologic storytelling. Repair fluency measures were not evaluated. Overall, the results support selective automation of clause-based fluency measurement, with human review concentrated on clause-end placement and related pause measures.
Authors
- Reiko Sato (ORCID: https://orcid.org/0009-0007-6038-7779)
- Jialiang Lu (ORCID: https://orcid.org/0000-0001-6162-1546)
Institutions
- Institute of Science Tokyo (JP)
Publication Details
- Journal
- Research Methods in Applied Linguistics
- Published
- 2026-11-14
- DOI
- https://doi.org/10.1016/j.rmal.2026.100397
- Primary Topic
- Linguistic Studies and Language Acquisition
- Type
- article
- Field-Weighted Citation Impact
- 0.00