Real-Time Noise Estimation for Lombard-Effect Speech Synthesis in Human–Avatar Dialogue Systems

To facilitate smooth communication between a human (client) and an avatar/robot, it is essential to generate avatar/robot speech that reduces the listening effort for the client in noisy acoustic environments. In this paper, we propose a framework for the following two purposes by leveraging the estimated noise using our previously proposed real-time blind speech extraction method: (i) real-time adjustment so that the power of the avatar/robot speech relative to that of the estimated noise is constant and sufficiently large, and (ii) mimicking the Lombard effect by simply applying filters to the avatar/robot speech. The proposed framework can also smoothly connect speeches generated from voice conversion (avatar) and text-to-speech (robot) techniques. Subjective evaluations demonstrated that the proposed framework achieves natural synthesis in noisy environments.