Multiple Apex Frame Detection and Emotion Analysis in Micro Expression Videos

Micro expressions are those minute fleeting expressions that appear on the human face, when subjected to an emotional stimulus. Their brevity poses a formidable challenge for both human observers and automated systems in discerning and interpreting them accurately. Recent research emphasizes the apex frame as the most significant image frame in a micro expression video, representing peak emotional intensity and stability in facial appearance. The apex frame captures the highest intensity of expression changes, while the onset frame serves as a reference with neutral expressions, while the offset frame indicates relaxation of muscular actions. The proposed method involves detection of multiple apex frames in a long video sequence and also classification of the emotions in the detected apex frames. The dataset used for this method is the widely popular CASME II dataset. Apex frame detection is performed using optical flow analysis and emotion analysis is performed using rule-based classification. The method also determines the indices and durations of the apex frames. Frame annotation provides visual feedback to the user. The paper also analyses the advantages, limitations and potential enhancements to this method.