Integrating Attention and Eulerian Magnification for Real Time Micro Expression and Heart Rate Detection

This research aims to detect micro-expressions and heart rate in real-time from human facial videos and explore the relationship between the two. Micro-expressions are very rapid and subtle facial expressions, crucial for lie detection and assessing mental health conditions. Heart rate can be monitored through variations in skin color caused by blood flow. The methodology includes using the CASME III dataset and developing two main models are the Micro-Expression Classification Model using Transformer architecture with Multi-Head Attention and Micro Attention techniques, and the Heart Rate Detection Model using Eulerian Video Magnification (EVM) and Fast Fourier Transform (FFT). The results indicate that the EVM method is effective for heart rate detection, achieving the lowest Mean Absolute Error (MAE) of 4.078 BPM. The Multi-Head Attention model shows the best accuracy of 99.05% and the lowest loss value of 0.07, while the Micro Attention model achieves 98.29% accuracy with a loss value of 0.14. The study finds that emotions reflected in micro-expressions can influence heart rate. Angry expressions tend to result in higher measurable heart rates, whereas sad expressions tend to result in lower measurable heart rates. Happy expressions show greater variation in measurement results.