Micro-Expression Classification With Weighted Locally Constrained Graph Network

Micro-expressions are non-verbal communication cues that are often brief and subtle that make their recognition difficult. Despite the progress made by deep learning-based methods, especially graph-based approaches, micro-expression classification still faces challenges in detecting their intricate features. Graph neural networks (GNNs) often suffer from oversmoothing problems, where node features become overly similar during iterative message passing, resulting in reduced discriminative capacity and an inability to comprehend the graph’s diverse structural information. To address these limitations, this paper introduces a novel weighted edge-node locally constrained graph attention network (WLC-GAT) which uses Dirichlet energy that is automatically applied to node features and global/local edge features within a graph attention network, ensuring feature distinctiveness for each class and preserving crucial information about the relationships between nodes and related edge structures. The paper utilizes a sliding window optical flow approach for the automated selection of high-intensity expression frames from videos and selects varying patch sizes across each landmark point determined by optical flow data to extract better features. To capture spatiotemporal information a three-frame graph structure is used. Finally, a dual-channel WLC-GAT network fusion focuses on discovering correlations among features. Empirical experiments demonstrate that the proposed approach consistently outperforms existing methods on three micro-expression databases (SAMM, CASME II, and SMIC) by significant margins on both single and composite databases.