Counterfactual discriminative micro-expression recognition

Abstract Micro-expressions are spontaneous, rapid and subtle facial movements that can hardly be suppressed or fabricated. Micro-expression recognition (MER) is one of the most challenging topics in affective computing. It aims to recognize subtle facial movements which are quite difficult for humans to perceive in a fleeting period. Recently, many deep learning-based MER methods have been developed. However, how to effectively capture subtle temporal variations for robust MER still perplexes us. We propose a counterfactual discriminative micro-expression recognition (CoDER) method to effectively learn the slight temporal variations for video-based MER. To explicitly capture the causality from temporal dynamics hidden in the micro-expression (ME) sequence, we propose ME counterfactual reasoning by comparing the effects of the facts w.r.t. original ME sequences and the counterfactuals w.r.t. counterfactually-revised ME sequences, and then perform causality-aware prediction to encourage the model to learn those latent ME temporal cues. Extensive experiments on four widely-used ME databases demonstrate the effectiveness of CoDER, which results in comparable and superior MER performance compared with that of the state-of-the-art methods. The visualization results show that CoDER successfully perceives the meaningful temporal variations in sequential faces.