The accuracy of automatic subtitles in Basque: a comparison between four speech-to-text programs

This contribution serves as an extension of a previously conducted study (Tamayo & Ros-Abaurrea, 2024) and delves into an in-depth analysis and comparison of the quality of Basque subtitles as generated by four speech-to-text recognition programs: Aditu, Etiqmedia, Speechmatics and Whisper. The study is based on a comprehensive examination of a corpus comprising six carefully selected samples, each spanning approximately 5 min in duration. All samples were gathered from news programs aired on the regional channel ETB1 and were selected based on a representativeness criterion. The methodology applied in this contribution combines both quantitative and qualitative approaches and involves an analysis based on the NER model. The three main foci of the quantitative study are the accuracy rate, error typology, and subtitle speed. From the analysis of the samples, it can be concluded that, despite none of the programs meeting the minimum threshold set by the NER model, platforms like Aditu or Speechmatics meet somewhat satisfactory levels of accuracy for a non-hegemonic language like Basque. In particular, the accuracy level met by Aditu stands out in comparison with the other programs, which could be linked to the fact that it is the only software specifically designed for Basque speech recognition.