The System of Visual Question Answering: Based on The Architectural Perspective

With the rapid development of computer vision and natural language processing technology, Visual Question Answering (VQA), as a cross-modal task, has gradually become a research hotspot in the field of artificial intelligence. The purpose of a VQA task is to enable a computer to understand the content of an image and accurately answer questions related to the image. The main challenge is how to effectively merge visual and verbal information to generate answers. In response to this challenge, the current research mainly focuses on the design of model architecture. At present, the VQA model architecture is composed of four parts: one is the visual encoder; Then there's the language encoder; Then multimodal fusion; and finally, the answer decoder. A lot of research and improvement has been done on these four parts. On the basis of a large number of literature studies, this paper firstly studies the VQA models of different architecture types in detail and classifies and summarizes their key steps. On this basis, the common data sets are introduced. In addition, the present problems of VQA are expounded and the future research direction is prospected.