MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering

Visual Question Answering (VQA) requires reasoning across visual and textual modalities, yet Large Vision-Language Models (LVLMs) often lack integrated commonsense knowledge, limiting their robustness in real-world scenarios.To address this, we introduce MAGIC-VQA, a novel framework that enhances VQA by systematically integrating commonsense knowledge with LVLMs.MAGIC-VQA employs a three-stage process: (1) Explicit Knowledge Integration from external sources, (2) By-Type Post-Processing for contextual refinement, and (3) Implicit Knowledge Augmentation using a Graph Neural Network (GNN) for structured reasoning.While GNNs bring greater depth to structured inference, they enable superior relational inference beyond LVLMs.MAGIC-VQA bridges a key gap by unifying commonsensse knowledge with LVLMdriven reasoning, eliminating the need for extensive pre-training or complex prompt tuning.Our framework achieves state-of-the-art performance on benchmark datasets, significantly improving commonsense reasoning in VQA.Our implementation is open-sourced on GitHub

Authors

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.findings-acl.872
Citations
1
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
1.28
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering

Shuo Yang, Eduard Hovy, Siwen Luo, Cheon Goo Han
1 citations
Multimodal Machine Learning Applications
1.28
article

MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering

Shuo Yang, Eduard Hovy, Siwen Luo, Cheon Goo Han
article en
1 citations

Abstract

Visual Question Answering (VQA) requires reasoning across visual and textual modalities, yet Large Vision-Language Models (LVLMs) often lack integrated commonsense knowledge, limiting their robustness in real-world scenarios.To address this, we introduce MAGIC-VQA, a novel framework that enhances VQA by systematically integrating commonsense knowledge with LVLMs.MAGIC-VQA employs a three-stage process: (1) Explicit Knowledge Integration from external sources, (2) By-Type Post-Processing for contextual refinement, and (3) Implicit Knowledge Augmentation using a Graph Neural Network (GNN) for structured reasoning.While GNNs bring greater depth to structured inference, they enable superior relational inference beyond LVLMs.MAGIC-VQA bridges a key gap by unifying commonsensse knowledge with LVLMdriven reasoning, eliminating the need for extensive pre-training or complex prompt tuning.Our framework achieves state-of-the-art performance on benchmark datasets, significantly improving commonsense reasoning in VQA.Our implementation is open-sourced on GitHub

Openalex Percentile: Top 18%
Multimodal Machine Learning Applications
1.28
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering — Shuo Yang, Eduard Hovy, et al. · (2025) | TGRS Research Map | TGRS