NeuralEVC: End-to-End High-Quality Expressive Voice Conversion

The voice conversion system has made significant progress in common voice conversion tasks, particularly in terms of performance regarding naturalness and similarity. However, challenges remain in more complex tasks such as expressive voice conversion. We designed a voice conversion model that is based on VITS and exhibits strong expressiveness to address these challenges. The model utilizes Wav2vec to extract phoneme posterior graph (PPG) features as content features that are speaker-independent, ensuring clarity of the speech content. Meanwhile, by incorporating speaker consistency loss and jointly training the speaker encoder, the model effectively captures the emotional and stylistic features of the target speech, ensuring accuracy in emotional expression during conversion. To further enhance the model’s performance, a lightweight SEANet-based neural decoder is introduced, significantly improving the quality and inference speed of voice conversion. Experimental results demonstrate that the proposed model outperforms existing baseline models in terms of speech naturalness and speaker similarity.