Adaptive Attention-based Unsupervised Domain Adaptation for Egocentric Action Recognition

In egocentric videos, collecting and annotating supervised data is more complicated and time-consuming than in exocentric videos, limiting research in this area. As a remedy, Unsupervised Domain Adaptation (UDA) enhances model performance on unlabeled target domains by bridging the distribution gap between source and target domains. However, UDA for egocentric action recognition is under-explored, facing unique challenges such as simultaneous learning of verb and noun representations, focusing on human-object interactions, and managing excessive verb-noun combinations. To tackle these issues, we propose a novel Unsupervised Domain Adaptation for Egocentric Action Recognition (UDA-EAR) approach that adaptively models egocentric actions and facilitates cross-domain knowledge transfer, improving recognition performance in unlabeled target domains. Specifically, our UDA-EAR employs adaptive spatio-temporal and spatio-channel attention in a dual-branch pipeline to focus on motion intervals and interaction regions, respectively, allowing specialized learning of discriminative representations while avoiding negative combination dependencies from domain gaps. Additionally, an adversarial domain alignment mechanism aligns the data distributions between source and target domains, effectively transferring fine-grained verb-noun knowledge of egocentric videos. Extensive experiments demonstrate that our UDA-EAR outperforms state-of-the-art baselines on widely used egocentric datasets, significantly improving egocentric action recognition accuracy. Our source codes and datasets are available at https://github.com/zou-y23/UDA-EAR.