Zh-Paral: Benchmark Dataset for Comprehension of Chinese Paralinguistic Speech

The substantial data present in spoken language is classified into two categories in our work: background and speaking. The speaking part is then divided into two subcategories: linguistic and paralinguistic. The majority of current speech LLMs prioritize the linguistic aspects of speech, with relatively little attention devoted to the processing of paralinguistic features. To evaluate and enhance the empathic capabilities of speech LLM, we present Zh-Paral, a benchmark dataset for the task of Chinese speech paralinguistic comprehension. This work is based on several open-source datasets and models. Following the application of a series of processing steps, including filtering, synthesizing, enhancement, and generating responses for the speech samples, a total of 11.5 hours of speech data was ultimately constructed. A three-phase incremental model was devised to assess the quality of the dataset, which encompasses three paralinguistic attributes with a total of 13 distinct labels. The output was scored by three large language models. The results demonstrate that Zh-Paral contains a wealth of paralinguistic data and responses that are both comprehensive and logical. Consequently, it serves as an exemplary benchmark dataset for evaluating the capacity of speech LLM to comprehend paralin-guistic information.