GAIP‐VAE: Balancing Reconstruction and Disentanglement in VAE With Group and Individual Priors

ABSTRACT Disentangled representation learning demonstrates great success in enhancing the explainability, robustness and generalization capability of models across computer vision domains. While adopting the variational auto‐encoders (VAEs) to learn disentangled representations holds great promise, these models are prone to suffer from the poor disentanglement capability in complicated datasets, for example, colourful portrait images. These datasets often contain strong correlation among attributes, making it difficult to disentangle them. To alleviate this issue, a novel approach named group and individual priors‐based VAE (GAIP‐VAE) is proposed, which constrains the semantic attributes by customizing prior information to improve the disentanglement capability of the VAE. Specifically, we start from modelling the joint distribution of the observed data, and then derive three compatible loss terms in the objective function. The first one is the reconstruction term, utilizing the Laplace distribution to improve the image quality. The second one is the individual prior regularizer, encouraging the model to learn more interpretable factors via dimensional‐level regularizer. The third one is the group prior regularizer, constraining the approximate posterior distribution through multivariate normal distribution with the tailored correlation. Both quantitative and qualitative experimental results demonstrate that GAIP‐VAE can achieve a great balance between image quality and disentanglement capability.