Dual-Branch Attention Transformer for Visual Commonsense Reasoning

Visual Commonsense Reasoning (VCR) is a challenging task that requires a model to select the correct answer in the context of a given image and question, while also providing a reasonable explanation to justify the chosen answer. In this paper, we propose a novel dual-branch attention network model that effectively models dense intra-modal and inter-modal interactions. Our model utilizes a self-attention mechanism to attend to visual and textual information independently, while employing a bidirectional guided attention mechanism to facilitate effective interaction between different modalities, thus correlating visual information with textual information. By stacking multiple dual-branch attention modules (DBAs), our model is capable of deeply exploring data features and extracting more powerful and informative prior features. By achieving a common focus on visual and textual information, we enhance the subtlety of our understanding and significantly improve our ability to perform visual reasoning. Our comprehensive and qualitative assessments on the public VCR dataset illustrate that our methodology surpasses most models, providing a new solution for Visual Commonsense Reasoning tasks.