Developing a digital tool for L2 speaking assessment in low-resourced languages

Previous research on training and assessment of oral skills has mainly focused on English as L2, but since languages and learning contexts vary, it is imperative to study automatic speaking assessment (ASA) in other languages with local relevance as well. This paper summarizes a project which set out to develop a prototype tool to support training and assessment of oral skills in two low-resourced languages, Finnish and Swedish. This project addressed the applicability of automated assessment to measure multiple features of monologue speech, the accuracy of human ratings used for training a system based on automatic speech recognition (ASR) and the technical conditions of providing individualized feedback to improve student learning. Encouraging results for both Finnish and Swedish were gained when adapting a big pre-trained wav2vec2.0 speech model that was fine-tuned first with a larger L1 dataset, and then with an L2 dataset collected in the project. The results suggested that the most suitable features for automatic analysis were quantifiable fluency measures and vocabulary range. Machine and human estimates were most consistent for assessing fluency, range and accuracy, while the results were more controversial for pronunciation features other than fluency. The prototype will be further developed, fine-tuned and adjusted to address the needs of adult learners preparing for the final test of integration training in L2 Finnish.