Cross-Granularity Relation-Aware Network for Visual Intention Understanding

Visual intention understanding (VIU) aims to decipher the inherent intention behind images related to human behavior. Intention reflects the psychological state of the person in or posting the image, requiring multi-level feature inference. The challenge arises from the assumption of independent and identically distributed data in deep neural networks, with the complex feature space in VIU exacerbating errors due to realworld distribution shifts. Since the hierarchical relation among the labels across different granularity levels is not constrained by this assumption, we explore such relation to guide the network. Specifically, we decompose the hierarchy across granularity levels into two relations in distinct directions. We introduce a crossgranularity relation-aware network (CGRAN), comprising two modules: a fine-based learnable embedding integration (FLEI) module at the feature layer and a coarse-based Hierarchical Calibration (CHC) module at the decision layer. Experimental results demonstrate the effectiveness of these modules and the superiority of our method over state-of-the-art VIU approaches.