Mask-Based Diffusion for Music Generation From Everyday Sounds

The application of artificial intelligence in music generation has opened up new avenues for creative expression, transforming how we interpret and produce music. This paper presents a novel methodology for generating music from everyday sounds using generative AI. We use a spectrogram based approach combined with masking in the diffusion process and prompt driven style conditioning. A key contribution of our research is the introduction of masking into the Img2Img pipeline for selective music generation. We present a feature that allows for custom mask generation based on an input spectrogram and hyper-parameters. Using this, we achieve fine-grained creative control in music generation, resulting in polished outputs that maintain the essence of the input sound while seamlessly integrating prompt-driven characteristics that specify the music style. Through this approach, we demonstrate a new direction for spectrogram-based music generation that combines feature retention with flexible stylistic adjustments, enabling a versatile framework for creative audio synthesis.