Micro-Expression Recognition Based on Twofold Feature Extraction Strategy and Dual-Convolutional Neural Network Architecture

Micro expressions are the key features to reveal concealed human emotions.Therefore, many application use micro-expression recognition (MiER) strategy as an interface for interactive computing.However, microexpressions are subtle and exist for very short period of time, making it extremely challenging to recognize accurately.Variety of approaches are designed to spot and identify micro-expressions automatically.Different types of feature extraction methods are designed and used for MiER.Many artificial intelligence-based approaches are also explored.However, improving MiER accuracy remains an open challenge with much scope for improvement in terms of finetuning various automatic MiER algorithms.Therefore, present work focuses on fine-tuning of feature extraction strategies by virtue of which betterment in automatic MiER and emotion classification is demonstrated.At the beginning dataset is pre-processed to sort region of interest with most relevant information by eliminate redundant details.Similarly, the motion magnification method is used to deal with subtleness and dataset insufficiency.To capture the subtle facial deformations, along with traditional 68 landmark points, additional 12 landmark points are considered covering cheek and near eyebrow region.The Dual-CNN architecture is formed to extract temporal information for automatic MiER.It consists of LM-CNN and OF-CNN, tuned to extract feature vectors based on 80 landmark points using strategies such as 2D landmark points feature map (2D-LPFM) and patches of optical flow respectively.The Output of Dual-CNN is strategically fused and used for MiER and emotion classification.The LOSO protocol is followed to test proficiency of the present work.The superior performance of the present framework in MiER task is validated with 81 % accuracy for SMIC dataset and 84 % accuracy for CASME II dataset.Moreover, robustness of this approach is tested on composite dataset and with UAR as 83 %, the proposed framework demonstrated superior performance.Thus, the 80 landmark points maximizes feature extraction from subtle facial deformations and two different types of feature extraction strategies provide enhanced robust MiER accuracy.