Facial expression recognition and the perceived emotions of three-to-five-year-old children while playing with apps

Despite the extensive literature on emotional expression recognition, few studies have focused on children. Current research has only reliably linked six prototypical emotions to specific combinations of ‘action units’ (AUs). The objective of this study is to assess whether thirteen emotional expressions automatically detected in three-to-five-year-old pupils during play correlated with an observer's perception of those emotions in context. We conducted a literature review of the expressions associated with AU combinations, extending beyond the basic emotions. Using a dataset of videos of spontaneous play by young children, we then conducted an exploratory study to test these sequences. The study provides a guide for the future analysis of automatic emotional facial expression recognition, a tool that could be groundbreaking for studying emotional behavior in children. However, this AI-based solution still contains certain biases, making it essential to enrich machine-learning results with human observations to ensure acceptable data quality. Since human behavior is context-related, only humans can interpret the activity in context, while machines are uniquely suited for a frame-level coding. Mutual learning between humans and machines can generate a co-evolutionary process, leading to progress in multidisciplinary research with children.