SMEM: A Spontaneous Micro-Expression Dataset in Museum

With the rapid growth of heritage tourism and increasing demands for museums to attract diverse audiences, understanding visitors’ emotional engagement and satisfaction has gained significant importance. Capturing and analyzing fleeting emotions and feelings in real-time provides critical insights into visitor behavior, offering opportunities to enhance visitor experiences, improve satisfaction, and support heritage conservation efforts. This underscores the need for innovative, technology-driven solutions to effectively evaluate and understand these emotional responses. Micro-expressions (MEs), fleeting and subtle FEs that reflect underlying emotions, are of significant importance for understanding human behavior. Automatic recognition of MEs is critical for applications that aim to discern true emotions, such as visitor experience analysis in museum settings. However, existing ME datasets are often limited to laboratory settings, which do not reflect the complexity and variability of real-world situations. To address this issue, we developed a novel spontaneous ME dataset in a museum setting (SMEM) comprising 127 participants and 344 ME video clips that were categorized into four basic types (negative, positive, surprise, and others). The onset frame, apex frame, offset frame, and action units (AUs) of each ME sample were annotated to accurately describe the facial movements. For baseline evaluation, we adopted optical flow to extract hand-crafted features of facial movements for ME apex spotting and designed a shallow CNN model with a cross-attention module to perform ME recognition with the optical flow as input. Our results show that the CNN method achieved a promising result for ME recognition in a real-world environment. The study provides a valuable contribution to the field of automatic ME recognition in real-world settings and highlights the potential of using ME recognition in museum visitor experience analysis.