A Multi‐Criteria Decision Framework for Performance Evaluation of Large‐Scale Research Infrastructure: A Fuzzy Borda Approach With IoT‐Enabled Case Study
ABSTRACT This study presents a multi‐criteria decision framework for quantitatively evaluating and enhancing the utilization efficiency of large‐scale instruments (LSIs) within academic institutions. Facing the complex socio‐technical challenge of balancing multi‐stakeholder needs, including researchers, facility managers, and institutional administrators, we developed an integrated evaluation model based on the fuzzy Borda method. The model systematically prioritizes four key system performance indicators: Total operating hours, Basic management capacity, Testing revenue, and Research output. Deployed in a real‐world case at Shanghai University involving 350 LSIs, indicators of Testing revenue and Research output show high sensitivity to Internet of Things (IoT) implementation in the presence of remote management and improved user access from the IoT‐based technology. This further leads to non‐uniform efficiency distributions under both part‐time and full‐time management regimes. Therefore, this framework was used to assess the impact of a pervasive IoT intervention. A significant 15.00% average improvement in overall system efficiency was achieved through implemented IoT. Monte Carlo simulation subsequently supports its robustness (between 14.87% and 14.98%). This work contributes a transferable decision‐support methodology for the operational performance evaluation of complex shared infrastructure, highlighting how the core challenges in system sustainment and optimization could be addressed via systematic evaluation and digital integration. By separating the measurement methodology contribution (a fuzzy Borda‐based evaluation framework) from the empirical IoT impact assessment, the study provides both a general‐purpose evaluation tool and concrete evidence of IoT's efficacy. Significance and Practitioner Points This study contributes a validated multi‐criteria decision framework by integrating the fuzzy Borda method with Monte Carlo‐based robustness analysis, capable of quantifying the utilization efficiency of shared research infrastructure. The framework systematically translates multi‐stakeholder preferences into weighted performance metrics, covering encompassing operational throughput, financial return, research output, and management quality; it also addresses inherent uncertainty and subjectivity in expert judgments. Empirical findings reveal that IoT‐driven efficiency gains are significantly moderated by underlying management regimes (full‐time versus part‐time staffing), offering new insights into the nature of technology‐enabled performance improvement in socio‐technical systems. For practitioners, the framework offers a transparent and actionable tool to assess high‐value shared assets. It converts initially ambiguous stakeholder priorities into clearly defined performance indicators, providing a solid basis for investment decisions and policy development. The Shanghai University case study illustrates how IoT deployment, including automated usage logging, QR‐code access, and dual‐sensor validation, can reduce administrative workload and improve instrument accessibility, thereby improving the utilization efficiency of shared research infrastructure. The framework can be adapted to various institutional settings, regardless of whether they operate under full‐time or part‐time management models, and allows stakeholder preferences to be updated periodically as institutional priorities shift over time.
Authors
- Haoliang Li (ORCID: https://orcid.org/0000-0002-8723-8112)
- Xinru Liu
Institutions
- Shanghai University (CN)
- Shanghai Research Institute of Materials (CN)
Publication Details
- Journal
- Systems Engineering
- Published
- 2026-08-24
- DOI
- https://doi.org/10.1002/sys.70087
- Primary Topic
- IoT and Edge/Fog Computing
- Type
- article
- Field-Weighted Citation Impact
- 0.00