Bernoulli priors as efficient denoising guides for diffusion models

We draw a connection between representation learning and efficient modelling in the diffusion domain. Particularly we relate the representation-learning capabilities of diffusion autoencoders (DAs) and efficient learning of diffusion models (DMs) that learn their forward process. DAs, through their input-dependent latent variables, can to varying extents be used for representation learning, controllable generation, and interpolation. However, their generative performance relies heavily on how well such a variable can be modelled and subsequently sampled from. DMs with learnable forward processes on the other hand are effective at adjusting the noise scales for the forward process in an input-dependent manner, but this means additional constraints derived from the terminal conditions of diffusion itself. We develop DDPM-BP, a diffusion model that employs a binary input-dependent variable as a bottleneck to guide the denoising process, satisfying the representation learning capabilities of DAs—evaluated on downstream tasks—as well as efficient learning and generation with fewer denoising steps compared to standard DMs.<br/>