A Fusion Framework for Micro-Expression Recognition Using Hierarchical Transformer Network with DEAC

Micro-expressions (MEs) are fleeting, involuntary facial movements that reveal genuine emotions often concealed during regular interactions.These expressions are challenging to recognize due to their brief duration, low intensity, and susceptibility to noise in visual data.To address these challenges, this paper proposes a novel framework for ME recognition, integrating Local Binary Patterns from Three Orthogonal Planes (LBP-TOP) on Tensor Independent Color Space (TICS), Hierarchical Transformer Networks (HTNet), and a Distributional Ensemble Actor-Critic (DEAC) reinforcement learning framework.LBP-TOP on TICS effectively decorrelates RGB components and extracts detailed spatio-temporal and color features, capturing both appearance and motion dynamics.These features are further processed by HTNet, which combines transformer layers and block aggregation layers to extract finegrained local details and coarse-grained global interactions across multiple scales, enhancing the detection of nuanced facial muscle movements.The final feature representations are classified using a Multi-Layer Perceptron (MLP) and optimized through the DEAC framework, which dynamically refines classification by employing reward-driven feedback to reduce misclassification errors.Experimental results demonstrate the proposed framework's superior performance on SMIC, CASME II, and SAMM datasets, achieving state-of-the-art accuracy of 98% and significant computational efficiency.