Schrödinger bridge-type diffusion models as an extension of variational autoencoders

Generative diffusion models are described by time-forward and -backward stochastic differential equations to connect the data and prior distributions. While conventional diffusion models (e.g., score-based models) only learn the backward process, more flexible frameworks have been proposed to also learn the forward process by employing the Schrödinger bridge (SB). However, due to the complexity of the mathematical structure behind SB-type models, we cannot easily give an intuitive understanding of their objective function. In this work, we propose a unified framework to construct diffusion models by reinterpreting the SB-type models as an extension of variational autoencoders. In this context, the data processing inequality plays a crucial role. As a result, we find that the objective function consists of the prior loss and drift matching parts, which enable us to reduce the numerical cost of training the forward process. Furthermore, we discuss the overfitting problem in the SB-type models in this framework.