Pioneering Facial Expression Generation from sEMG Signals with Diffusion Models

Facial expressions play an important role in human communication by conveying a wide range of emotions and intentions. While recent advances in generative models have enabled the synthesis of facial expressions from various prompts-including text, images, and audio-these approaches often overlook the direct physiological basis of facial movements. In this work, we present the first framework that uses surface electromyography (sEMG) signals, recorded from facial muscles during expression tasks, to guide a denoising diffusion probabilistic model (DDPM) for facial expression image generation. Our method employs a CNN-LSTM network to extract spatial-temporal features from sEMG signals, which are then integrated with an identity frame (a neutral facial image of the target subject) to serve as a conditional prompt for the DDPM. Experimental results demonstrate that sEMG signals can effectively guide the generation of realistic and expressive facial images, achieving a Frechet Inception Distance (FID) of 96.67, Learned Perceptual Image Patch Similarity (LPIPS) of 0.08, and an expression recognition accuracy of $100 \%$. These findings highlight the potential of sEMG signals as a novel condition prompt for generative models. This work pioneers the use of sEMG signals in facial expression synthesis, offering a new paradigm for applications in human-computer interaction, virtual reality, and medical rehabilitation, where precise and personalized expression control is critical.