Enhancing Facial Expression Recognition With AI Agents: A Semisupervised Guided Adaptive $\beta$-VAE Coupled With Interval Type-2 Fuzzy Classifier

Facial Expression Recognition (FER) is a complex task, hindered by subtle distinctions between expression classes, significant variability within each class, and external influences like identity, pose, age, and ethnicity. As a result, achieving pure expression encodings that are resilient to exogenous factors proves elusive, thereby compromising the downstream classification tasks. This study presents a novel intelligent facial expression recognition scheme that mitigates the impact of external confounders by integrating disentangled representation learning with fuzzy logic. Building on Adaptive <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">β</i>-VAE [1] as a backbone, we develop a semi-supervised Guided Adaptive <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">β</i> Variational Autoencoder (GA-<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">β</i>-VAE) capable of isolating expression features from exogenous factors. Specifically, the adaptive <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">β</i>-VAE is augmented with two additional branches: a deformable PCA-based secondary decoder that disentangles expression-irrelevant transformations from the core expression content, and an adversarial excitation–inhibition branch that forces the “target” (expression) latent variables to be informative only of expressions. This yields well-separated, expression-centric embeddings that are subsequently processed by an Interval Type-2 (IT2) fuzzy classification unit to predict the corresponding expression classes. By avoiding reliance on paired data or explicit annotations, this approach offers a scalable and flexible solution for FER. Experimental evaluations on benchmark datasets (CK+, FER+, and RAF-DB) demonstrate the framework’s effectiveness in addressing the challenges posed by exogenous factors, achieving superior accuracy and interpretability compared to state-of-the-art methods.