Design and Implementation of Emotional Speech Synthesis Algorithm for Tibetan Virtual Digital People

This article first constructs an emotion classification system applicable to Tibetan, a minority language with unique phonetic characteristics and rhythmic rules, covering six emotions: joy, sadness, anger, surprise, fear, and neutrality. Using a combination of recurrent neural network (RNN) and long short-term memory (LSTM), sentiment features are extracted from Tibetan text and classified through fully connected layers. In the acoustic feature mapping stage, the fundamental frequency (F0), speech speed, volume and timbre are defined and adjusted, and emotional tags are mapped to acoustic features by using linear regression and MLP model. In order to further improve the naturalness of emotional expression, a prosody generation algorithm is designed, and the strategy of generating fundamental frequency curve and adjusting its length is optimized. In the stage of acoustic modeling and waveform synthesis, the Transformer model is used to predict the acoustic characteristics from sequence to sequence (Seq2Seq), and the WaveNet model is used to realize high-quality speech waveform synthesis. The experimental results show that the synthesized speech is excellent in naturalness and fidelity of emotional expression, especially in happiness and sadness. The research in this paper not only enriches the technical accumulation in the field of Tibetan speech synthesis, but also provides a useful reference for the study of emotional speech synthesis in other minority languages.

Authors

Publication Details

Published
2024-10-18
DOI
https://doi.org/10.1109/icdacai65086.2024.00166
Primary Topic
Speech and Audio Processing
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Design and Implementation of Emotional Speech Synthesis Algorithm for Tibetan Virtual Digital People

Ting Luo, Xi Han, Kai Liang
Speech and Audio Processing
article

Design and Implementation of Emotional Speech Synthesis Algorithm for Tibetan Virtual Digital People

Ting Luo, Xi Han, Kai Liang
article en

Abstract

This article first constructs an emotion classification system applicable to Tibetan, a minority language with unique phonetic characteristics and rhythmic rules, covering six emotions: joy, sadness, anger, surprise, fear, and neutrality. Using a combination of recurrent neural network (RNN) and long short-term memory (LSTM), sentiment features are extracted from Tibetan text and classified through fully connected layers. In the acoustic feature mapping stage, the fundamental frequency (F0), speech speed, volume and timbre are defined and adjusted, and emotional tags are mapped to acoustic features by using linear regression and MLP model. In order to further improve the naturalness of emotional expression, a prosody generation algorithm is designed, and the strategy of generating fundamental frequency curve and adjusting its length is optimized. In the stage of acoustic modeling and waveform synthesis, the Transformer model is used to predict the acoustic characteristics from sequence to sequence (Seq2Seq), and the WaveNet model is used to realize high-quality speech waveform synthesis. The experimental results show that the synthesized speech is excellent in naturalness and fidelity of emotional expression, especially in happiness and sadness. The research in this paper not only enriches the technical accumulation in the field of Tibetan speech synthesis, but also provides a useful reference for the study of emotional speech synthesis in other minority languages.

Openalex Percentile: Top 72%
Speech and Audio Processing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Design and Implementation of Emotional Speech Synthesis Algorithm for Tibetan Virtual Digital People — Ting Luo, Xi Han, et al. · (2024) | TGRS Research Map | TGRS