AGILE: Attribute-Guided Identity Independent Learning for Facial Expression Recognition

Within computer vision, Facial Expression Recognition (FER) is a challenging task involving bottlenecks such as obtaining well-separated and compact expression embeddings that are invariant of identity. This study introduces AGILE (<b>A</b>ttribute-<b>G</b>uided <b>I</b>dentity-Independent <b>Le</b>arning), an innovative approach to enhance FER by distilling identity information and promoting discriminative features. Initially, an adaptive <inline-formula><tex-math notation="LaTeX">$\beta$</tex-math></inline-formula> Variational Autoencoder (VAE) is proposed from a fixed <inline-formula><tex-math notation="LaTeX">$\beta$</tex-math></inline-formula>-VAE architecture leveraging the theory of single-dimensional Kalman filter. This enhances disentangled feature learning without compromising the reconstruction quality. Now, to achieve the desired FER objective, we design a two-stage modular scheme built within the framework of adaptive <inline-formula><tex-math notation="LaTeX">$\beta$</tex-math></inline-formula>-VAE. In the first stage, an expression-driven identity modeling is proposed where an identity encoder is trained with a novel training loss to embed the most likely state corresponding to every subject in latent representation. In the next stage, keeping the identity encoder fixed, an expression encoder is trained with explicit guidance for the latent variables using an adversarial excitation and inhibition mechanism. This form of supervision enhances the transparency and interpretability of the expression space and helps to capture discriminative expression embeddings required for the downstream classification task. Experimental evaluations demonstrate that AGILE outperforms existing methods in identity and expression separability in the latent space and achieves superior performance over state-of-the-art methods on both lab-controlled and in-the-wild datasets, with recognition accuracies of 99.00% on CK+, 90.00% on Oulu-CASIA, 89.01% on MMI, 67.20% on Aff-Wild2, and 68.97% on AFEW.