Disentanglement Learning of Facial Expression and Appearance

Abstract Facial expression analysis holds great promise beyond conventional emotion recognition, particularly in medical diagnostics and emotional well-being. However, disentangling tightly intertwined factors, such as facial expression and appearance, remains a significant challenge. This study presents a framework utilizing StyleGAN inversion and SimCLR to focus on expression-specific attributes while systematically minimizing appearance-related factors. Despite the implementation of tailored augmentations like Style-Mixing and Latent Space Slider, the disentanglement of expression from appearance was not fully successful. Residual appearance information persisted in the learned representations, as shown by clustering dominated by appearance rather than expression in t-SNE visualizations. However, the accuracy of emotion classification reached 95.82 % with augmented CNNs, demonstrating the potential of this approach. These findings highlight the limitations of current disentanglement techniques and the need for further refinement to achieve robust separation of expression and appearance. Advancing this work could enhance applications in emotion recognition and privacy-preserving medical diagnostics.