Instruction-tuned QwenChart for Chart Question Answering

Charts, where information is delivered jointly by visual and textual features, represent a challenge when it comes to downstream tasks such as chart question answering, where both kinds of information contribute to the task.The standard approach is to decouple the task in two steps, first extracting information from the charts, or representing it as a table, text or code, and then a second reasoning step to output the answers.Today, the advancements in visual encoding of Visual Large Language Models (VLLM) have shown their capabilities to solve such complex tasks without using in-between representations of the charts or massive in-domain training.We propose a solution for the Scientific Visual Question Answering (SciVQA) Shared Task, on which our team THAii_LAB scored the second position in the final leaderboard.Our new instruction fine-tuned and Chain-of-Thought (CoT) model QwenChart-7B showed that even in a complex new benchmark general models can achieve great performances with low-cost training, matching the capabilities that LLMs have showed in unimodal downstream tasks.An out-of-domain evaluation showed satisfactory results, albeit with an expected drop in performance.