Text-image pair joint enhancement VQA model based on cross-modal attention

In light of the issues such as polarized inference results and unreliable feature fusion caused by inadequate interaction between visual features and question textual features in existing Visual Question Answering (VQA) methods, this paper introduces an attention mechanism to enhance feature representation through intra-modal self-attention mechanism and to strengthen text-image feature interaction via inter-modal cross-attention mechanism. Additionally, the image class label feature from the same feature space as the question textual feature is introduced as an anchor, the performance of intermodal alignment is significantly improved. Finally the effectiveness of our method is validated through a series of experiments on VQA-v2 and VQA-CP-v2 with accuracy metric. Compared to the referenced alternative methods, our model attains superior performance across both Overall and Other types in the VQA-v2 and VQA-CP-v2 datasets. Additionally, it demonstrates competitive results in other types of tasks, thereby validating the efficacy of our approach.