Intelligent Decision Support System Design for Research Assessment Exercise

This paper proposes a framework of Generative Artificial Intelligence (GenAI)-based Decision Support System (DSS) named MetricAI tailored to the Hong Kong Research Assessment Exercise (RAE). The designed conceptual framework aims to automate the evaluation of researchers' publications across three core dimensions: originality, significance, and rigor by leveraging state-of-the-art GenAI technologies. Beyond providing quantitative scores and personalized feedback, the system enables researchers to make better decisions by integrating these evaluation results with additional supporting information, such as journal impact factors and altmetrics, thereby optimizing their research performance and RAE submission strategies. The conceptual system architecture consists of four layers: data ingestion, AI processing, decision support, and user interface. This paper focuses on proposing the architectural design of such a GenAI-based DSS, and through a preliminary interface design case study, we demonstrate how the GenAI-based DSS can provide data-driven insights through a human-machine collaborative approach, thereby improving evaluation efficiency. The MetricAI is designed to improve efficiency, transparency, and strategic nature of RAE preparation work.