A statistical journey to DNN, the fifth trip: riding variational autoencoder

Variational Autoencoder (VAE) is a foundational method used in generative deep neural networks that has significantly contributed to recent advances in artificial intelligence.However, VAE is challenging to understand since its theoretical underpinnings involve complex statistical concepts.This paper elucidates how VAE operates by providing a systematic and accessible overview of the statistical foundations of VAE.It presents VAE as a generalization of reduced-rank regression and factor regression, and revisits the EM algorithm to interpret the meaning of ELBO which is the objective function of VAE.It concludes by discussing variational inference, amortized inference, the architecture of VAE, and implementation strategies to provide deeper insights into VAE.