Exploring the Relevance and Societal Impact of Artistic Research using an LLM based RAG tool

Artistic research occupies an ambiguous position in Nordic research policy: societal impact is asserted in national frameworks and institutional strategies but is rarely operationalised into assessable criteria, and artistic outputs remain largely invisible to standard bibliometric infrastructures. This presentation reports on ongoing work, conducted as part of Work Package 4 of COST Action CA23158 “Artistic Intelligence,” developing a retrieval-augmented generation (RAG) interface over a self-harvested mirror of the Research Catalogue (https://www.researchcatalogue.net) comprising over 6,500 expositions of documented artistic research over the last 15 years. The tool allows evidence of societal engagement, relevance, and effect to be retrieved and synthesised across the corpus at a scale unavailable to manual review, addressing the discoverability problem identified as the central obstacle to using this material for research assessment. The tool has already been tested against the Times Higher Education Sustainability Rankings’ SDG classifier, with promising results in surfacing societally relevant expositions across the corpus. Rather than treating the interface as a scoring mechanism, the project asks what conditions must hold for such retrieval to support legitimate evaluation of societal impact: what counts as evidence, whose judgements are represented, and what is lost when heterogeneous artistic outputs are rendered searchable. The research-policy payoff lies in demonstrating how a large-scale database of artistic research can be systematically explored to extract evidence of its societal relevance, informing future Nordic approaches to evaluating artistic research. A working prototype is available at (password needed): https://eirikursmari.github.io/react-hee2rtww/