Micro-Expression Key Frame Inference

Micro-expressions (MEs) are brief, involuntary facial movements critical for detecting lies, drawing growing interest in psychology and computer science. However, annotating ME can burden human coders with excessive time commitment and overwhelming information that compromises coding reliability and efficiency. Such difficulties in data annotation also led to the small sample size problem and hindered the development of ME analysis. Specifically, our psychological research highlights the complexities involved in human annotation of key frames. To facilitate the annotating process of ME, we proposed the Micro-Expression Key Frame Inference (ME-KFI) problem, aiming to identify MEs’ temporal locations from a single frame, reducing manual annotation effort. We propose a Micro-Expression Contrastive Identification Annotation (MECIA) method as a solution to ME-KFI, including three modules: a contrastive module, an identification module, and an annotation module, corresponding to the three steps of manual annotation. The network’s outputs infer the key frame of ME clips. MECIA demonstrates superior performance over random baselines on SAMM and CAS(ME)<inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> databases and maintains comparable recognition accuracy with ground-truth clips.