Question-Aware Reasoning Framework via Two-Level Cross-Attention

Multimodal chain-of-thought (CoT) reasoning has emerged as a pivotal research direction in artificial intelligence. However, current approaches predominantly adopt a linear CoT structure with a single reasoning module and complex multi-level gated multi-hop cross-attention mechanisms for modality fusion, which exhibit notable limitations. Specifically, the inability of linear CoT structures to dynamically select appropriate reasoning modules based on problem characteristics often leads to hallucinations during intermediate reasoning. Moreover, tightly coupled gating and cross-attention mechanisms can inadvertently suppress critical information flow during inter-modal interactions, resulting in erroneous predictions. To address these challenges, we propose a novel multimodal reasoning framework, M-TCM, that integrates a two-level cross-attention fusion mechanism with a single-level gating strategy. This design not only reduces the complexity of modality fusion but also effectively preserves information crucial for intermediate reasoning. Furthermore, M-TCM incorporates a novel module selection strategy. We first construct a new dataset, SQ-GPT4, to complement the existing ScienceQA dataset and facilitate the training of two distinct reasoning modules. Subsequently, the model dynamically selects the most appropriate reasoning module for prediction based on the specific skill requirements of each problem. Experimental results on the ScienceQA benchmark demonstrate the superiority of our proposed model, achieving a prediction accuracy of 88.23%.