Visual Question Answering Based on Object Detection and Recognition

Visual question answering (VQA) is a relatively recent research area that infers answers to image queries. Significant work is proposed because of its numerous real-world applications and growing interest. However, there is still a lack of object-based study for VQA. To bridge this gap, we proposed a novel VQA framework based on object detection and recognition. The presented methodology works on the selectively target visual regions of an object. The proposed VQA covers the four subproblems: object detection, object recognition, attribute detection, and counting. In this research, we performed VQA based on the three different perspectives of object detection, attribute detection and top hierarchy generation. YOLOv8 is used for object detection and k-means for attribute detection. The experiments are conducted with humans interacting with common objects named HICO dataset. The training set is used to generate elements of top hierarchy and the test set is used to evaluate the proposed method. Our proposed work attained the performance of 84%.