Automatic cross-lingual annotation of clause-based utterance fluency in L2 speech

Automatic annotation of temporal speech features can scale second language (L2) fluency research. However, existing approaches are validated mainly on English and vary in accuracy across measures. We present a cross-lingual pipeline that computes utterance fluency measures from audio and operationalizes the Vercellotti and Hall (2024) clause framework. We validate it on 40 L2 English and 40 L2 Japanese storytelling monologs. Evaluation follows a staged design: clause-boundary agreement, pause location classification, and concurrent validity with manual reference measures. Reference clause annotations combined independent double coding, large language model-assisted segmentation, coder review, and an audio check. In both languages, the clause boundaries placed by the pipeline agreed well with the manual reference. The pipeline’s classification of pauses as mid-clause or end-clause agreed closely with the reference in English and well in Japanese. All nine automatic measures correlated strongly with the manual measures in both languages. In a matched English comparison, the pipeline measured articulation rate and pause ratio with much smaller errors than existing acoustic and transcription-based tools. Among the clause-based measures, mid-clause pause ratio gained the most over the transcription-based tool. Speech rate, mean length of run, and overall pause ratio were the most robust measures. End-clause pause ratio was the weakest. These findings are limited to monologic storytelling. Repair fluency measures were not evaluated. Overall, the results support selective automation of clause-based fluency measurement, with human review concentrated on clause-end placement and related pause measures.

Authors

Institutions

Publication Details

Journal
Research Methods in Applied Linguistics
Published
2026-11-14
DOI
https://doi.org/10.1016/j.rmal.2026.100397
Primary Topic
Linguistic Studies and Language Acquisition
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Automatic cross-lingual annotation of clause-based utterance fluency in L2 speech

Reiko Sato, Jialiang Lu
Research Methods in Applied Linguistics
Linguistic Studies and Language Acquisition
article

Automatic cross-lingual annotation of clause-based utterance fluency in L2 speech

Reiko Sato, Jialiang Lu
article en

Abstract

Automatic annotation of temporal speech features can scale second language (L2) fluency research. However, existing approaches are validated mainly on English and vary in accuracy across measures. We present a cross-lingual pipeline that computes utterance fluency measures from audio and operationalizes the Vercellotti and Hall (2024) clause framework. We validate it on 40 L2 English and 40 L2 Japanese storytelling monologs. Evaluation follows a staged design: clause-boundary agreement, pause location classification, and concurrent validity with manual reference measures. Reference clause annotations combined independent double coding, large language model-assisted segmentation, coder review, and an audio check. In both languages, the clause boundaries placed by the pipeline agreed well with the manual reference. The pipeline’s classification of pauses as mid-clause or end-clause agreed closely with the reference in English and well in Japanese. All nine automatic measures correlated strongly with the manual measures in both languages. In a matched English comparison, the pipeline measured articulation rate and pause ratio with much smaller errors than existing acoustic and transcription-based tools. Among the clause-based measures, mid-clause pause ratio gained the most over the transcription-based tool. Speech rate, mean length of run, and overall pause ratio were the most robust measures. End-clause pause ratio was the weakest. These findings are limited to monologic storytelling. Repair fluency measures were not evaluated. Overall, the results support selective automation of clause-based fluency measurement, with human review concentrated on clause-end placement and related pause measures.

Research Methods in Applied LinguisticsVol. 5(3)
Institute of Science Tokyo (JP)
Linguistic Studies and Language Acquisition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon