English-to-Hindi Speech-to-Speech Translation of Interviews with Automatic Voice Switching

An interview serves as an effective means of acquiring knowledge from professionals in their respective fields. These recorded interviews utilizing an online platform can benefit larger audiences seeking knowledge in a specific area. It is quite prevalent for experts to converse in English during these interviews, which might occasionally result in a loss of interest among non-English listeners or those who are not proficient in English. The existing Speech-to-Speech Translation (S2ST) models translate source speech into target speech in a single canonical voice. This study introduces a pipeline for S2ST that converts recorded interview speech from English to Hindi with an Indian accent. Within the pipeline, automatic detection of male and female speakers occurs, and subsequently, their speech is translated into male and female canonical voices. The S2ST model is composed of an automatic speech recognizer (ASR), machine translator (MT), and text-to-speech synthesis (TTS), along with gender voice recognition and voice switching.