Post Training Quantization Strategies for Diffusion Models

This study addresses the critical need for quantizing audio diffusion models to enable efficient synthesis on low-resource-constrained devices. We propose to design objective audio quality metrics for realism and fidelity, which can be utilized to optimize the denoising process of diffusion models. The current state of our research includes the application of quantization strategies from the image domain to off-the-shelf audio diffusion models such as AudioLDM and Make-an-Audio. The proposed work consists of exploring the specific operations in the U-Net architecture that should be quantized, focusing on weights and activations and how to effectively calibrate them. We plan to evaluate our framework on mobile devices to cater to a range of hardware- off the shelf and customized.