Towards a Robust Group-Level Emotion Recognition via Uncertainty-Aware Learning

Group-level emotion recognition (GER) is an inseparable part of human behavior analysis, aiming to recognize an overall emotion in a multi-person scene. However, the existing methods are devoted to combing diverse emotion cues while ignoring the inherent uncertainties under unconstrained environments, such as congestion and occlusion occurring within a group. Additionally, since only group-level labels are available, inconsistent emotion predictions among individuals in one group can confuse the network. In this paper, we propose an uncertainty-aware learning (UAL) method to extract more robust representations for GER. By explicitly modeling the uncertainty, we adopt stochastic embedding sourced from a Gaussian distribution instead of deterministic point embedding. It helps capture the probabilities of emotions and facilitates diverse inferences. Additionally, we adaptively assign uncertainty-sensitive scores as the fusion weights for individuals’ faces within a group. Moreover, we developed an image enhancement module to evaluate and filter samples, strengthening the model’s data-level robustness against uncertainties. The overall three-branch model, encompassing face, object, and scene components, is guided by a proportional-weighted fusion strategy and integrates the proposed uncertainty-aware method to produce the final group-level output. Experimental results demonstrate the effectiveness and generalization ability of our method across three widely used databases.