Accentedness ratings of Brazilian speakers’ L2 English by LLMs: Systematic upward shift from the human baseline

Abstract This study examines whether audio-capable large language models (LLMs) align with human patterns of L2 English accentedness judgment. Google Gemini Flash and Gemini Pro rated the L2 English speech of Brazilian Portuguese speakers from the CSLU Foreign Accented English Corpus ( Lander, 2007 ) and were compared against native American English listeners. Both models assigned systematically more severe ratings, using the highest category of the rating scale at frequencies not observed in the human data. The shift was most pronounced in Gemini Pro ( OR = 8.25), suggesting that greater model capacity does not yield closer alignment with human judgment. The two models also failed to agree with each other. These divergences raise concerns about the validity of LLM accentedness ratings as proxies for human judgment.

Authors

Institutions

Publication Details

Journal
Journal of Second Language Pronunciation
Published
2026-11-13
DOI
https://doi.org/10.1075/jslp.26063.kup
Primary Topic
Phonetics and Phonology Research
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Accentedness ratings of Brazilian speakers’ L2 English by LLMs: Systematic upward shift from the human baseline

Felipe Flores Kupske, Laura Zorzi
1 links
Journal of Second Language Pronunciation
Phonetics and Phonology Research
article

Accentedness ratings of Brazilian speakers’ L2 English by LLMs: Systematic upward shift from the human baseline

Felipe Flores Kupske, Laura Zorzi
article en

Abstract

Abstract This study examines whether audio-capable large language models (LLMs) align with human patterns of L2 English accentedness judgment. Google Gemini Flash and Gemini Pro rated the L2 English speech of Brazilian Portuguese speakers from the CSLU Foreign Accented English Corpus ( Lander, 2007 ) and were compared against native American English listeners. Both models assigned systematically more severe ratings, using the highest category of the rating scale at frequencies not observed in the human data. The shift was most pronounced in Gemini Pro ( OR = 8.25), suggesting that greater model capacity does not yield closer alignment with human judgment. The two models also failed to agree with each other. These divergences raise concerns about the validity of LLM accentedness ratings as proxies for human judgment.

Journal of Second Language Pronunciation
Universidade Federal do Rio Grande do Sul (BR)
Phonetics and Phonology Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon