Sound effect synthesis from language-independent mimicry: a case study on explosion sounds

Existing vocal-imitation-based methods for environmental sound or sound effect synthesis typically use root-mean-square envelopes, text prompts or discrete codes, and do not directly learn spectral correspondences between imitation and target sounds. This study uses vocal mimicry, nonlinguistic imitation produced by the human articulatory system, as a control signal for sound effect synthesis. This study proposes PronounSE, a method that directly learns spectral correspondences between vocal mimicry and sound effects to generate outputs whose nuances follow the mimicry input. To address data scarcity, this study constructs a new paired vocal-mimicry data set focused on explosion sounds as a case study, since this category exhibits diverse acoustic variations and is relatively amenable to vocal mimicry due to its predominantly aperiodic, noise-like characteristics. In addition to naturalness and audio quality, this study evaluates mimicry-driven synthesis with two task-specific criteria: fidelity to the target sound and reflection of vocal mimicry nuances. These four criteria support both objective and subjective evaluation. Results show strong objective-subjective correlations for fidelity and confirm that PronounSE effectively reflects mimicry nuances. Results also indicate that fidelity is strongly affected by mimicry skill and speaker-specific characteristics, highlighting the importance of evaluating fidelity and nuance reflection as distinct criteria in vocal-mimicry-based sound effect synthesis.