Visual Question Answering on Scientific Charts Using Fine-Tuned Vision-Language Models

Scientific charts often encapsulate the core findings of research papers, making the ability to answer questions about these charts highly valuable.This paper explores recent advancements in scientific chart visual question answering (VQA) enabled by large Vision Language Models (VLMs) and newly curated datasets.As part of the SciVQA shared task from the 5th Workshop on Scholarly Document Processing, we develop and evaluate multimodal systems capable of answering diverse question typesincluding multiple-choice, yes/no, unanswerable, and infinite answer set questions -based on chart images extracted from scientific literature.We investigate the effects of zero-shot and one-shot prompting, as well as supervised finetuning (SFT), on the performance of Qwen2.5-VLmodels (7B and 32B variants).We also tried to include more training data from domainspecific datasets (SpiQA and ArXivQA).Our fine-tuned Qwen2.5-VL32B model achieves a substantial improvement over the GPT-4o mini baseline and reaches the 4th place in the shared task, highlighting the effectiveness of domainspecific fine-tuning.We published the code for the experiments 1 .