ManFolkSyn: an end-to-end model for Mandarin folk song singing voice synthesis

Singing voice synthesis (SVS) has advanced to the point where it can produce high-quality synthetic voices based on input text and musical scores. However, current SVS research predominantly focuses on synthesizing pop songs, with little attention given to other musical genres. This paper presents an end-to-end folk song singing voice synthesis model to address this gap. Firstly, we constructed a Mandarin folk song singing voice dataset named Folk107, which comprises 107 Mandarin folk songs and nursery rhymes. These songs were recorded in professional settings, resulting in a total duration of approximately three hours. Then, we developed a fully end-to-end model for Mandarin folk singing voice synthesis, named ManFolkSyn. Finally, we conducted both SVS and singing voice conversion (SVC) experiments. In the SVS experiments, MOS scores for two singers exceeded 3.60, while in the SVC experiments, similarity scores surpassed 4.00. These results demonstrate the utility of the dataset and the effectiveness of the model we proposed.

Authors

Institutions

Publication Details

Published
2024-10-28
DOI
https://doi.org/10.1117/12.3050029
Primary Topic
Music and Audio Processing
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

ManFolkSyn: an end-to-end model for Mandarin folk song singing voice synthesis

Hongwu Yang, Weizhao Zhang, Yu Zuo
Music and Audio Processing
article

ManFolkSyn: an end-to-end model for Mandarin folk song singing voice synthesis

Hongwu Yang, Weizhao Zhang, Yu Zuo
article en

Abstract

Singing voice synthesis (SVS) has advanced to the point where it can produce high-quality synthetic voices based on input text and musical scores. However, current SVS research predominantly focuses on synthesizing pop songs, with little attention given to other musical genres. This paper presents an end-to-end folk song singing voice synthesis model to address this gap. Firstly, we constructed a Mandarin folk song singing voice dataset named Folk107, which comprises 107 Mandarin folk songs and nursery rhymes. These songs were recorded in professional settings, resulting in a total duration of approximately three hours. Then, we developed a fully end-to-end model for Mandarin folk singing voice synthesis, named ManFolkSyn. Finally, we conducted both SVS and singing voice conversion (SVC) experiments. In the SVS experiments, MOS scores for two singers exceeded 3.60, while in the SVC experiments, similarity scores surpassed 4.00. These results demonstrate the utility of the dataset and the effectiveness of the model we proposed.

Northwest Normal University (CN)
Openalex Percentile: Top 79%
Music and Audio Processing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

ManFolkSyn: an end-to-end model for Mandarin folk song singing voice synthesis — Hongwu Yang, Weizhao Zhang, et al. · (2024) | TGRS Research Map | TGRS