How Does Alignment Error Affect Automated Pronunciation Scoring in Children's Speech?

Automated goodness of pronunciation scores measure deviation from typical adult speech by first phonetically segmenting speech using forced alignment and then computing phoneme likelihoods. Care must be taken to distinguish between the impact of alignment error (a spurious signal) and true acoustic deviation on the automated score. Using mixed effects modeling, we predict <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>Δ</mml:mi> <mml:mi>P</mml:mi> <mml:mi>L</mml:mi> <mml:mi>L</mml:mi> <mml:mi>R</mml:mi></mml:math> , the difference between pronunciation scores computed using manual alignment ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>P</mml:mi> <mml:mi>L</mml:mi> <mml:mi>L</mml:mi> <mml:msub><mml:mrow><mml:mi>R</mml:mi></mml:mrow> <mml:mrow><mml:mi>m</mml:mi></mml:mrow> </mml:msub> </mml:math> ) versus computed using automatic forced alignments ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>P</mml:mi> <mml:mi>L</mml:mi> <mml:mi>L</mml:mi> <mml:msub><mml:mrow><mml:mi>R</mml:mi></mml:mrow> <mml:mrow><mml:mi>a</mml:mi></mml:mrow> </mml:msub> </mml:math> ). Pronunciation deviations and alignment error are both magnified in children's speech and may be influenced by factors such as phoneme position and phoneme type. Our methodology shows that alignment error has a moderate effect on <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>Δ</mml:mi> <mml:mi>P</mml:mi> <mml:mi>L</mml:mi> <mml:mi>L</mml:mi> <mml:mi>R</mml:mi></mml:math> , and other variables have small to no effect. Manual <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>PLLR</mml:mi></mml:math> closely matches automatically calculated <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>PLLR</mml:mi></mml:math> following cross utterance averaging. Thus, practical comparisons between child speakers should be very comparable across the two methods.