Enhancing Training Speed and Stability Through Symmetry in Diffusion Models

Diffusion model has achieved great success in generative models, but they require significant time and data to train. Furthermore, the training of the diffusion model can be unstable or fail to converge, especially when using an inappropriate learning rate with limited GPU capacity. We suggest a Symmetrize Noise Sampling (SNS) method to achieve faster and more stable convergence of the diffusion models especially in limited computing environments. This method guarantees that the mean of the noise sampled from the Gaussian distribution used to train the diffusion model is always zero, thereby ensuring that the mathematical assumptions of the diffusion model are strictly adhered to throughout the training process. The experimental results have validated the effectiveness of our method on training of a diffusion model.