Information Extraction from Chart Figures in Scientific Articles for the ORKG

Information in the ORKG is mostly derived from text in the scientific articles. However, it does not utilize the information in the figures. Inferring insights from the charts is challenging: Chart figures contain text and visual elements to convey meaning through visual encoding and representation of data such as points, lines, bars etc. In related studies, there is a lack of research in context and domain-aware chart comprehension. Our objective is to address these challenges, utilizing state-of-the-art vision-language models. The envisioned results will enhance scientific communication, for instance making the contents of scientific visualizations accessible to visually impaired readers; and contribute to the development of the ORKG leading to more comprehensive listings of paper contributions and comparisons. Finally, the ORKGAsk could be improved viachart-based insights, with references to the figures in question, surrounding text paragraphs and additional domain knowledge.