Examining Prosodic Elements in Speech-to-Speech Machine Translation: Focusing on Stress Transfer in Indian Context

This study primarily aims to enhance an English-to-Hindi Speech-to-Speech Machine Translation (SSMT) system that can improve the delivery of educational content by incor-porating stress cues into the translated speech and increasing listener engagement and interest. Frame-level features such as FO, energy, Mel Frequency Cepstral Coefficients and Shifted Delta Coefficients are extracted and fed into StressNet, a Time Delay Neural Network-based stress detection model, which evaluates the presence of stress in individual frames, subsequently synthesizing these frame-level assessments into an aggregated word-level prediction. To enable the incorporation of stress in the generated speech in the target language, variance predictors inspired by FastPitch are added to the YourTTS-based end-to-end Text-to-Speech (TTS) architecture. Comprehensive evaluations were con-ducted on both the stress detection model and the modified TTS system, which helped identify the most effective configurations for achieving optimal performance. The paper demonstrates the effectiveness of this approach, presenting a refined English-to-Hindi SSMT system that enhances stress transfer ability in the target speech.