What is the True Performance of Large Multimodal Models in Visual Context-Based Mathematical Reasoning? An Analysis of Multiple Datasets and Future Research Directions

In this paper, we study the capability of visual context-based mathematical reasoning within the rapidly evolving field of Large Multimodal Models (LMMs). Achieving visual context-based mathematical reasoning requires cognitive skills similar to those used in human reasoning, as it involves inter-preting diverse visual elements and converting them into mathe-matical representations. Previous research has explored various methodologies to address this challenge, but these approaches tend to work only under specific conditions and are often constrained by the limitations of available datasets. As a result, this paper offers a comprehensive analysis of datasets related to visual context-based mathematical reasoning and evaluates the effectiveness of LMMs using these datasets. We aim to identify the limitations of existing techniques and suggest future research directions.