FreeSVC: Towards Zero-shot Multilingual Singing Voice Conversion
This work presents FreeSVC, a promising multilingual singing voice conversion approach that leverages an enhanced VITS model with Speaker-invariant Clustering (SPIN) for better content representation and the State-of-the-Art (SOTA) speaker encoder ECAPA2. FreeSVC incorporates trainable language embeddings to handle multiple languages and employs an advanced speaker encoder to disentangle speaker characteristics from linguistic content. Designed for zero-shot learning, FreeSVC enables cross-lingual singing voice conversion without extensive language-specific training. We demonstrate that a multilingual content extractor is crucial for optimal cross-language conversion. Our source code and models are publicly available<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>.
Authors
- Alef Iury Siqueira Ferreira (ORCID: https://orcid.org/0000-0002-9119-6357)
- Lucas Rafael Stefanel Gris (ORCID: https://orcid.org/0000-0002-2099-5004)
- Anderson S. Soares (ORCID: https://orcid.org/0000-0001-9701-0819)
- Edresson Casanova (ORCID: https://orcid.org/0000-0003-0160-7173)
- Rafael Sousa (ORCID: https://orcid.org/0000-0001-9394-522X)
- Frederico Santos de Oliveira (ORCID: https://orcid.org/0000-0002-5885-6747)
- Alessandro Pasquale De Rosa (ORCID: https://orcid.org/0000-0001-5165-3114)
- Arlindo Rodrigues Galvão Filho (ORCID: https://orcid.org/0000-0003-2151-8039)
- Arnaldo Cândido (ORCID: https://orcid.org/0000-0002-5647-0891)
Institutions
- Universidade Tecnológica Federal do Paraná (BR)
- Universidade Federal de Mato Grosso (BR)
- Universidade Federal de Goiás (BR)
- Universidade Estadual Paulista (Unesp) (BR)
Publication Details
- Published
- 2025-03-12
- DOI
- https://doi.org/10.1109/icassp49660.2025.10890068
- Primary Topic
- Speech Recognition and Synthesis
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Research and Development