Statistics-Driven Texture Sound Synthesis Using Differentiable Digital Signal Processing-Based Architectures

This thesis investigates the application of Differentiable Digital Signal Processing (DDSP) architectures to the synthesis and control of texture sounds. Texture soundsynthesis presents unique challenges due to the complex, non-pitched and noisy nature of these sounds, which differ significantly from pitched instrument timbres forwhich DDSP was originally designed. The core of this research consists in developing novel differentiable signal processors and loss functions tailored to the characteristicsof texture sounds, and integrating these innovations into DDSP-based frameworks.Two synthesizers are introduced: the SubEnv Synth, which imposes amplitude envelopes in a subband decomposition of white noise; and the P-VAE Synth, which combines a Poisson process with a Variational Autoencoder (VAE) to deal with time and events respectively. This approach is directly inspired by Saint-Arnaud’s original conception of texture sounds. Additionaly, the TextStat loss function, designed to compare texture sounds based on their statistical properties rather than short-term perceptual similarity, is also presented. This loss function is strongly inspired by McDermott and Simoncelli’s work.The thesis details the implementation of two DDSP-based architectures incorporating these synthesizers and the TextStat loss function. Results demonstrate that the SubEnv Synth successfully resynthesizes texture sounds with some succes, while the P-VAE Synth faces learning challenges. These challenges might be related to its VAE module’s complexity, however more research is needed to make this claim. Additionally, this research explores model variations that show potential for improved performance and suggests future work including the optimization of the TextStat loss function, reconsideration of VAE use, and real-time implementation of the developed models.Overall, the contributions of this thesis provide a foundation for advancing texture sound synthesis, offering new tools and insights for both theoretical exploration andpractical applications in the field of audio signal processing.