A Transformer-Enabled Phonological Study of Genuine and Synthetic Voices

This study bridges the gap between traditional linguistic studies, particularly phonology, and modern Machine Learning (ML) developments. It explores the mutually beneficial interactions between the two domains: by utilizing Speech-To-Text (STT) models to aid in phonology and by integrating phonological insights into the evaluation of Text-To-Speech (TTS) generation. Phonology, the study of phonemes-fundamental sound units in human speech-is pivotal for understanding speech variations caused by accents, speech disorders, and individual pronunciation differences. Leveraging the International Phonetic Alphabet (IPA), this work analyzes speech patterns to enhance explainability and accuracy in TTS models, particularly those employing transformer architectures. This approach for phoneme analysis is tested on the Speech Accent Archive and synthetic speech samples generated by OpenAI's text-to-speech system. By employing the wav2vec model for phoneme detection and DeepPhonemizer for grapheme-to-phoneme conversion, we calculate phoneme distances using an adapted Wagner-Fischer algorithm. This analysis reveals significant differences in phoneme accuracy between native English speakers, non-native speakers, and synthetic speech, highlighting areas for improvement in synthetic speech generation. The results indicate that synthetic voices exhibit lower phoneme variation compared to human speakers, suggesting potential for enhancing both the naturalness and diversity of synthetic speech. This approach not only aids in refining TTS systems but also offers insights into developing more unique and personalized synthetic voices. Future work aims to refine ML tools, expand the phonological approach, and further integrate linguistic knowledge into TTS evaluation and improvement.