Visual Question Answering based on Multimodal Deep Feature Fusion

To solve the problem of shallow cross-modal interaction depth, information loss during feature fusion, and semantic ambiguity leading to low algorithm accuracy in the field of visual question answering, a visual question answering algorithm based on multimodal deep feature fusion (MDFF-VQA) is proposed. The algorithm uses Self-Attention unit (SA) to enhance question features and Guided-Attention unit (GA) to iteratively extract relevant visual features. Improve the feature fusion by using 1 ×1 convolution to expand dimensions and overcome size limitations, enriching the fused features through fully connected layers. Experiments on the dataset VQA v2 show that MDFF - VQA significantly outperforms algorithms such as Language-only, MLAN, Scence GCN, and ODA-GCN in “Y/N”, “Num”, “Other”, and “All” type questions, effectively improving the performance of visual question answering tasks.