Generative AI through non-Gaussian "diffusion"

Generative diffusion models present a powerful framework for image generation, but their reliance on Gaussian noise assumptions may limit their effectiveness in denoising tasks involving non-Gaussian noise. In this study, we empirically investigate whether the Gaussian noise assumption is essential for the functionality of diffusion models or if it can be replaced by a Poisson distribution. We approach this by modifying an existing Stable Diffusion implementation and developing a simplified one-dimensional toy model to simulate the denoising process using both Gaussian and Poisson noise. Our results demonstrate that while the diffusion model retained its functionality, albeit with more variance. These findings suggest potential for broader noise modelling in diffusion-based models, but highlight that special care needs to be taken to safeguard the model against null collapse through additional methods or parametrization.