When Metrics Fall Short: Building a Values-Based Evaluation Framework

Arts & Humanities and Social Sciences (AHSS) research evaluation remains a persistent challenge in research assessment. Existing indicator-based approaches are largely derived from STEM-oriented databases and evaluation frameworks, with limited visibility for AHSS outputs, practices, impacts. This creates difficulties both for researchers seeking recognition for their breadth of contributions and institutions, funders, governments seeking robust evidence of research value amid increasing pressure to demonstrate societal, educational, economic impact. Our work was informed by findings from a qualitative study of 17 semi-structured interviews with scholars and research administrators across seven countries. Interviewees identified widespread concerns regarding reliance on STEM-derived indicators, lack of community-endorsed metrics, mismatch between evaluation cycles and longer timescales for research influence and impact development. We propose a framework which starts from values and contributions that AHSS scholars consider meaningful and seeks indicators to serve as evidence. We used wide variety of data sources to operationalise this into Values-based Scholarship Array (VBSA) for evaluative settings. VBSA turns each value and audience into a set of evidence tiles, applicable to individuals at different career stages, supported by qualitative evidence as well as quantitative indicators. The resulting evidence can be aggregated to research groups, disciplines, departments, universities or national systems. VBSA could help institutions, evaluators, scholars assess AHSS scholarship fairly and transparently. It aligns indicators with dimensions including knowledge advancement, community membership, collaboration, collegiality, leadership, education, and engagement. These indicators should complement, not replace, narrative assessment and expert peer review, enabling an equitable, context-sensitive, scalable evaluation of AHSS research and impact.